Apache Ranger compiled and installed

Open Source Developer Notes: DevOps, microservices, distribution, Big data, high availability, blockchain, whitepapers, Algorithms, hacking, design patterns, interview questions. O star ⭐ ️

Apache Ranger is a centralized security management framework in the field of big data. It aims to implement centralized security management of Hadoop components by formulating policies. Users can use Ranger to securely access data in the cluster.

The environment

The framework	version
Ubuntu	16.04
JAVA	OpenJDK-8-jdk
Python	2.7
Maven	3.6.3
Ranger	2.0.0
Hadoop	3.1.3
Solr	8.5.2

The source code to compile

Environment to prepare

Download Ranger

Wget https://downloads.apache.org/ranger/2.0.0/apache-ranger-2.0.0.tar.gz or use mirror wget https://mirrors.tuna.tsinghua.edu.cn/apache/ranger/2.0.0/apache-ranger-2.0.0.tar.gzCopy the code

Download Maven

Wget HTTP: / / https://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gzCopy the code

Install Java, python, and c++ compilation environments

Apt install openjdk-8-jdk apt install python2.7 apt install GCC g++ ln -s /usr/bin/python2.7 /usr/bin/pythonCopy the code

Configuring environment Variables

exportMAVEN_HOME = / opt/app/apache maven -- 3.6.3export PATH=$PATH:$MAVEN_HOME/bin
export MAVEN_OPTS=-Xmx2048m
Copy the code

Use the Ali Maven source

/ opt/app/apache maven - 3.6.2 / conf/Settings. The XML

<mirrors>
   <mirror>
     <id>nexus-aliyun</id>
     <mirrorOf>central</mirrorOf>
     <name>Nexus aliyun</name>
     <url>http://maven.aliyun.com/nexus/content/groups/public</url>
   </mirror>

   <mirror>
     <id>CN</id>
     <name>OSChina Central</name>
     <url>http://maven.oschina.net/content/groups/public/</url>
     <mirrorOf>central</mirrorOf>
   </mirror>

   <mirror>
     <id>alimaven</id>
     <mirrorOf>central</mirrorOf>
     <name>aliyun maven</name>
     <url>https://maven.aliyun.com/nexus/content/repositories/central/</url>
   </mirror>

   <mirror>
     <id>jboss-public-repository-group</id>
     <mirrorOf>central</mirrorOf>
     <name>JBoss Public Repository Group</name>
     <url>https://repository.jboss.org/nexus/content/groups/public</url>
   </mirror>
</mirrors>
Copy the code

compile

The tar xf apache - ranger - 2.0.0. Tar. GzcdApache-ranger-2.0.0mvn clean compile package assembly:assembly install-dskiptests-drat.skip =trueOpen the target folder apache-Ranger-2.0.0 /target# lltotal 1605220 drwxr-xr-x 5 root root 4096 Jun 29 11:04 ./ drwxr-xr-x 59 root root 4096 Jun 29 03:20 .. / drwxr-xr-x 2 root root 4096 Jun 29 03:20 antrun/ drwxr-xr-x 2 root root 4096 Jun 29 11:04 archive-tmp/ drwxr-xr-x 3 root root 4096 Jun 29 03:20 maven-shared-archive-resources/ -rw-r--r-- 1 root root 30 Jun 29 11:04 .plxarc -rw-r--r-- 1 Root root 248635508 Jun 29 11:01 Range-2.0.0-admin.tar. gz-rw-r --r-- 1 Root root 249666712 Jun 29 11:02 Range-2.0.0-admin.zip -rw-r--r-- 1 root root 27787895 Jun 29 11:03 Range-2.0.0-atlas-plugin.tar. gz -rw-r--r-- 1 root Root 27832012 Jun 29 11:03 Range-2.0.0-atlas-plugin. zip -rw-r--r-- 1 root root 31555304 Jun 29 11:04 Rang-2.0.0-elasticsearch-plugin.tar. gz -rw-r--r-- 1 root root 31605841 Jun 29 11:04 Zip -rw-r--r-- 1 root root 26638377 Jun 29 10:58 Ranger-2.0.0-hbase-plugin.tar.gz -rw-r--r-- 1 root root 26665982 Jun 29 10:58 Ranger-2.0.0-hbase-plugin. Zip -rw-r--r-- 1 root root 23971392 Jun 29 10:58 Rw-r --r-- 1 root root 23997623 Jun 29 10:58 Range-2.0.0-hdFs-plugin. zip -rw-r--r-- 1 Root root 23825995 Jun 29 10:58 Range-2.0.0-hive-plugin.tar. gz-rw-r --r-- 1 root root 23854522 Jun 29 10:58 Rw-r --r-- 1 root root 39930681 Jun 29 10:59 Range-2.0.0-kafka-plugin.tar. gz-rw-r --r-- 1 Root root 39983878 Jun 29 10:59 Rang-2.0.0-kafka-plugin. zip -rw-r--r-- 1 root root 90982883 Jun 29 11:02 Range-2.0.0-kms.tar. gz -rw-r--r-- 1 root root 91106270 Jun 29 11:02 Range-2.0.0-kms.zip -rw-r--r-- 1 root root 28380704 Jun 29 10:58 Range-2.0.0-knox-plugin.tar.gz-rw-r --r-- 1 root root 28411022 Jun 29 10:58 Range-2.0.0-knox-plugin. zip -rw-r--r-- 1 root root 23940874 Jun 29 11:03 Range-2.0.0-kylin-plugin.tar.gz-rw-r --r-- 1 Root root 23980053 Jun 29 11:03 Range-2.0.0-kylin-plugin. zip -rw-r--r-- 1 root root 34223 Jun 29 11:02 Range-2.0.0-migration-util.tar. gz -rw-r--r-- 1 root root 37740 Jun 29 11:02 Range-2.0.0-migration-util.zip -rw-r--r-- 1 root root 26388071 Jun 29 11:00 Range-2.0.0-ozone - plugin.tar.gz-rw-r --r-- 1 root root 26421136 Jun 29 11:00 Rw-r --r-- 1 root root 40302042 Jun 29 11:04 Range-2.0.0-presto-plugin.tar.gz-rw-r --r-- 1 root root 40341626 Jun 29 11:04 Range-2.0.0-presto-plugin. zip -rw-r--r-- 1 root root 22232050 Jun 29 11:02 Range-2.0.0-range-tools.tar.gz-rw-r --r-- 1 root root 22248747 Jun 29 11:02 Range-2.0.0-range-tools.zip -rw-r--r-- 1 Root root 42667 Jun 29 11:02 Ranger-2.0.0-solr_audit_conf.tar. gz -rw-r--r-- 1 root root 45636 Jun 29 11:02 Range-2.0.0-solr_audit_conf.zip -rw-r--r-- 1 root root 26964416 Jun 29 11:00 Range-2.0.0-solr-plugin.tar.gz -rw-r--r-- 1 root root 27010058 Jun 29 11:00 Ranger-2.0.0-solr-plugin. zip -rw-r--r-- 1 root root 23952732 Jun 29 11:03 Rw-r --r-- 1 root root 23986050 Jun 29 11:03 Range-2.0.0-sqoop-plugin. zip -rw-r--r-- 1 Root root 4081388 Jun 29 11:03 Range-2.0.0-src.tar. gz-rw-r --r-- 1 Root root 6257752 Jun 29 11:03 Range-2.0.0-src.zip -rw-r--r-- 1 root root 37230628 Jun 29 10:59 Range-2.0.0-storm-plugin.tar. gz -rw-r--r-- 1 root root 37268719 Jun 29 10:59 Range-2.0.0-storm-plugin. zip-rw-r --r-- 1 root root 32772036 Jun 29 11:02 Range-2.0.0-tagsync.tar.gz-rw-r --r-- 1 root root 32782070 Jun 29 11:02 Range-2.0.0-tagsync. zip -rw-r--r-- 1 root root 16256778 Jun 29 11:02 Range-2.0.0-usersync.tar. gz -rw-r--r-- 1 root root 16280886 Jun 29 11:02 Range-2.0.0-usersync.zip -rw-r--r-- 1 root Root 23953849 Jun 29 10:59 range-2.0.0-yarn-plugin.tar. gz-rw-r --r-- 1 root root 23992362 Jun 29 10:59 Range-2.0.0-yarn-plugin.zip -rw-r--r-- 1 root root 5 Jun 29 11:04 versionCopy the code

An error occurred while compiling

C++ compilation environment is not installed

[INFO] Unix Native Authenticator .......................... FAILURE [s] 1.387 [INFO] -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 08:51 min [INFO] Finished at: 2020-06-030T12:00:43-02:00 [INFO] Final Memory: 96M/420M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal Org.codehaus. mojo:native maven-plugin:1.0-alpha-8:compile (default-compile) on project credValidator: Error executing command line. Exit code:127 -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:native maven-plugin:1.0-alpha-8:compile (default-compile) on project credValidator: Error executing command line. Exit code:127Copy the code

Don’t install python

[INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal Org, apache maven. Plugins: maven - antrun - plugin: 1.7: the run () generate - version - an annotation on project ranger - util: An Ant BuildException has occured: exec returned: 1 [ERROR] around Ant part ... <exec failonerror="true" executable="python">... @4:48 in /data/ apache-range-2.0.0 / range-util /target/antrun/build-main. XML [ERROR] -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org, apache maven. Plugins: maven - antrun - plugin: 1.7: the run () generate - version - an annotation on the project ranger-util: An Ant BuildException has occured: exec returned: 1 around Ant part ... <exec failonerror="true" executable="python">... @ so in/data/apache - ranger - 2.0.0 / ranger - util/target/antrun/build - the main XMLCopy the code

Install the ranger – admin

Unpack the

Tar ZXVF Ranger-2.0.0-admin.tar. gz -c /opt/ranger CD /opt/rangerCopy the code

Modify the ranger – 2.0.0 – admin/install. The properties

# database connectionSQL_CONNECTOR_JAR = / opt/ranger/mysql connector - Java - 8.0.18. Jar db_root_user = root db_root_password = root db_host = localhost db_name=ranger db_user=ranger db_password=rangeradmin# log audit
audit_store=solr
audit_solr_urls=http://localhost:6083/solr/ranger_audits
audit_solr_user=solr
Copy the code

Run ranger-2.0.0-admin/setup.sh to install it

Success is output. Properties For solr, mysql, and mysql driver packages, see the following

[I] Checking Connection passed. Installation of Ranger PolicyManager Web Application is completed.Copy the code

Run range-2.0.0-admin /set_globals.sh to configure the soft link

Run the ranger-admin start or ranger-2.0.0-admin/ews/ranger-admin-services.sh start command to start the service

Visit http://ip:6080/ admin/admin

Install solr

Modify the ranger – 2.0.0 – admin/contrib/solr_for_audit_setup/install. Properties

SOLR_INSTALL=true

JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
SOLR_DOWNLOAD_URL=https://mirrors.tuna.tsinghua.edu.cn/apache/lucene/solr/8.5.2/solr-8.5.2.tgz

SOLR_INSTALL_FOLDER=/data/solr   Install directory, this is optional
SOLR_RANGER_HOME=/data/solr/ranger_audit_server
SOLR_RANGER_PORT=6083   # default port
SOLR_DEPLOYMENT=standalone  # Deployment mode (single node deployment), and SolrCloud (cluster) mode
SOLR_RANGER_DATA_FOLDER=/data/solr/ranger_audit_server/data   # Data storage directory
Copy the code

Perform ranger – 2.0.0 – admin/contrib/solr_for_audit_setup/setup. Sh

Tue Jun 30 06:58:31 UTC 2020|INFO|Installed Solr in /opt/solr Tue Jun 30 06:58:31 UTC 2020|INFO|Configuring standalone instance Tue Jun 30 06:58:31 UTC 2020|INFO|Copying Ranger Audit Server configuration to /opt/solr/ranger_audit_server Tue Jun 30 06:58:31 UTC 2020|INFO|Creating group solr Tue Jun 30 06:58:31 UTC 2020|INFO|Creating user solr Tue Jun 30 06:58:31 UTC 2020|INFO|Done configuring Solr for Apache Ranger Audit Tue Jun 30 06:58:31 UTC 2020|INFO|Solr HOME for Ranger Audit is /opt/solr/ranger_audit_server Tue Jun 30 06:58:31 UTC 2020|INFO|Data folder for Audit logs is /opt/solr/ranger_audit_server/data Tue Jun 30 06:58:31 UTC 2020|INFO|To start Solr run /opt/solr/ranger_audit_server/scripts/start_solr.sh Tue Jun 30 06:58:31 UTC 2020|INFO|To stop Solr run /opt/solr/ranger_audit_server/scripts/stop_solr.sh Tue Jun 30 06:58:31 UTC 2020|INFO|After starting Solr for RangerAudit, it will listen at 6083. E.g http://weihai-2:6083 Tue Jun 30 06:58:31 UTC 2020|INFO|Configure Ranger to use the following  URL http://weihai-2:6083/solr/ranger_audits Tue Jun 30 06:58:31 UTC 2020|INFO| ** NOTE: If Solr is Secured then solrclient JAAS configuration has to be added to Ranger Admin and Ranger Plugin properties Tue Jun 30 06:58:31 UTC 2020|INFO| ** Refer documentation on how to configure Ranger for audit to Secure Solr########## Done ###################
Created file /opt/solr/ranger_audit_server/install_notes.txt with instructions to start and stop
## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
Copy the code

Start/stop Solr

/opt/solr/ranger_audit_server/scripts/start_solr.sh
/opt/solr/ranger_audit_server/scripts/stop_solr.sh
Copy the code

MySQL installation

Download the mysql driver package at https://downloads.mysql.com/archives/c-j/

Download the mysql server

Official website to download

https://dev.mysql.com/downloads/mysql/
Copy the code

To update the APT source

Wget https://dev.mysql.com/get/mysql-apt-config_0.8.15-1_all.de PKG -i mysql - apt - config_0. 8.15 1 _all. Deb apt search mysql-server Sorting... Done Full Text Search... Done Auth2db/Xenial 0.2.5-2+ DFSG-5 Ubuntu1 All Powerful and eyecandy IDS Logger, Log Viewer and Alert Generator mysql-server/unknown,now 8.0.20-1Ubuntu16.04 AMd64 [installed] mysql Server Meta Package depending on latest version apt install mysql-serverCopy the code

Configure the mysql

create user 'ranger'@'%' identified by 'rangeradmin';
create database ranger;
grant all privileges on ranger.* to 'ranger'@'%';
set global log_bin_trust_function_creators=TRUE;
SET GLOBAL innodb_lock_wait_timeout=500;
Copy the code

Apache Ranger compiled and installed

The environment

The source code to compile

Environment to prepare

compile

An error occurred while compiling

Install the ranger – admin

Install solr

MySQL installation

Related Posts

Lock&Condition for concurrency tools

Simple micro-service continuous integration solution, SpringBoot+Docker+Jenkins+Gitlab

Even if deleted the whole library, how to recover half an hour?