This article is based on the learning progress of MaxCompute product, combined with some questions in the developer community, and then sorted out the paper. Hope to help you.

How do I check the time when a data item is synchronized to MaxCompute after data source incremental synchronization? You can view the LastModifiedTime time of the table metadata, but you cannot view the time of a specific record.

Question 2. Can the developed data processing scripts be migrated in different project Spaces? Cross-project cloning can only be performed with the same account and region, so project backup and recovery can be used between different regions. You can refer to the official document: help.aliyun.com/document_de… Tips: This function supports file recovery at the present stage, the file size must be less than 30M, the later version will be updated, please continue to pay attention to.

When creating a workspace, the client will display “Project does not exist”, or switch the workspace to show “Project does not exist”. What is the reason? Generally, this problem is caused by the failure to select the MaxCompute compute engine service during the project construction after the account is opened. The project can be rebuilt to solve the problem. The correct choices are as follows.



Tips: Pay by volume and package annual and monthly options are required to purchase services. The developer version is available as a free trial. Developers opening address: promotion.aliyun.com/ntms/act/cl…

If MaxCompute is used as the big data platform to access the data of the whole business system, mysql, SQLServer and other data sources are available. When the source database table, field changes, there is no good monitoring method. For example, the field length becomes longer, the table increases the field, and so on DDL operations. Because the service is changing, if the monitoring is not timely, will the whole scheduling process collapse? Currently, MaxCompute only supports viewing and monitoring internal data changes in the DataWorks console data map. Field changes in external data sources are currently not monitored. If there is a good solution, we will inform the community, please continue to pay attention to.

Q5. How to batch query table metadata? On August 13, Qu Ning, product expert of Alibaba Cloud intelligence business group, broadcast the latest feature introduction of Alibaba’s big data products — MaxCompute, in which an Information Schema metadata service provides quasi-real-time metadata in the form of view. Includes Tables, PartitionsColumns, UDFs, Resources, Users, Roles, Packages, authorization information of data objects, and job running information. You can submit a work order to test the new function. Please refer to help.aliyun.com/document_de…

When Pyodps is used to calculate the null value rate of a field in a table, is it recommended to use Execute_SQL for SQL statements or DataFrame for aggregate operations? Which has higher performance? DataFrame aggregation performs better, so it is recommended to use DataFrame to perform aggregation operations. For details on how to aggregate DataFrame, see help.aliyun.com/document_de…

Question 7. How do I locate the MaxCompute error message? The common error information of MaxCompute is defined in specifications. The exception number of MaxCompute is general description. The error messages of SQL, MapReduce, and Tunnel are different. Some error messages are listed in the official documentation. You can query the corresponding error messages: help.aliyun.com/document_de…

How do cross-business process dependencies exist in DataWorks? This can be resolved as a node, where the first node of the downstream business process depends on the last node of the upstream business process.

How to solve the problem that the MaxCompute client (ODPSCMD) queries Chinese garbled characters in the Windows command line? Odpscmd in Chinese garbled because Chinese character set encoding of the inconsistent, document can refer to a practice to solve the problem: yq.aliyun.com/articles/70…

Question 10: How do I determine whether a field is null in MaxCompute Sql? IS NULL. MaxCompute Sql contains relational, arithmetic, bitwise, and logical operators. Please refer to the official documentation: help.aliyun.com/document_de…

Author: Kang Haipeng

The original link

This article is the original content of the cloud habitat community, shall not be reproduced without permission.