Dynamic description of Aliyun e-MapReduce
-
E-mapreduce 2.3.1 Image Version (Released)
- Upgrade CentOS 6.5 kernel version to 2.6.32-642
- Hadoop YARN Job failover
information
- Big data winter has arrived, who will fall, who will become a giant?
Based on the recent news of percentage points and internal personnel adjustment of CICA data, this paper puts forward the view that “the winter of big data has arrived”, and excessive competition is the main factor of the winter of big data. As for how big data companies will survive in the future, the article puts forward directions such as product focus and cost control to prepare for the winter.
- Why did Weibo win the first big data case?
Weibo won the case of “Maimai illegally captures the information of users using Weibo”. The author believes that Maimai mainly involves the following aspects: 1. Obtain user information illegally and use it for commercialization; 2. 2. The behavior constitutes unfair competition; 3, Mai Mai does not play the role of protecting user information. This case also serves as a warning to the industry: it is the responsibility of all platforms to promote the prosperity of the data ecosystem, actively establish rules for the use of data, and prevent the abuse and excessive use of data.
- Apache Software Foundation has announced Apache Eagle as a top category
Apache Software Foundation has officially announced that Eagle has graduated from the Apache Incubator program and is officially upgraded to a top-level program. Eagle is an open source big data distributed real-time monitoring and warning solution of eBay, which has been applied to eBay, Paypal, Yihaodian and other companies. Open source big data solutions represented by Hadoop are evolving towards security, stability, observability and other enterprise-level requirements.
technology
- Pear Video: Practice of building video recommendation system based on Aliyun e-MapReduce
Pear video is a traditional media entrepreneurial short video software, and emerging in the field of video, this paper introduces how to use ali cloud to quickly build core data platform and recommendation system, realize the business, the system USES ECS, OSS, SLS, EMR, Redis, RDS, completing a full range of product data flow through.
- Jd’s real-time big data computing platform based on Docker
The article introduced the problems encountered in the internal use of Storm platform in JINGdong Company, such as diverse and complex user resource demands, large cluster maintenance, cost saving, etc., and how to use Docker technology to transform Storm to meet the requirements of user application, personalized configuration, large-scale cluster, efficient and automatic operation.
- Intel open source distributed deep learning library BigDL: Supports high-performance big data analysis
Intel has opened source a distributed deep learning library, BigDL, that runs on Apache Spark to run deep learning computations using existing Spark clusters and simplify data loading from Hadoop’s large datasets. Tests on Xeon servers showed that BigDL achieved significant speed improvements over open source frameworks such as Caffe, Torch or TensorFlow. Its speed is comparable to that of mainstream Gpus
- Performance evaluation of Hadoop 3.0 Erasure codes
One of the main functions added in the new version of Hadoop 3.0.0-Alpha1 is the erasure code technology. This paper first briefly introduces the erasure code technology, and then mainly evaluates the performance of the erasure code technology, as well as the performance comparison between the erasure code technology and HDFS default 3 backup technology.