Ssl urls hive os что это

fsck /dev/sda2
но шансов на успех немного. Даже если и поможет — через пару тройку дней флешка сдохнет окончательно. Причем по закону Мерфи, она обязательно сделает это тогда, когда заменить ее возможности не будет.

wanted92

Свой человек

HaloGenius

Друг форума

вызов майнера miner выход Ctrl+A+D
в логе EthMIner есть только периодические отчеты, опциями командной строки можно расширить информации. Кнопок как у клемора или феникса нет

HaloGenius

Друг форума

0.6-156@200904

CИСТЕМА
Улучшена поддержка стороннего оборудования (добавлена поддержка новых контроллеров вентиляторов от CoolBox)
Улучшена процедура применения разгона после команды hello
Исправлено поведение agent при команде смены пароля на риге

МАЙНЕРЫ
⛏ CryptoDredge v0.25.1 (исправлена проблема высокой закрузки процессора на алгоритме MTP при получении нового задания, улучшены MTP и Argon2d-NIM алгоритмы, добавлена возможность задания интенсивности с плавающей точкой, удалены некоторые устаревшие алгоритмы, исправлены ошибки)
⛏ T-Rex v0.16.2 (исправление ошибок: ‘can’t init enumerate, code -6’ при применении твиков памяти на P102/104/106 cards, твики памяти подвисали после прекращения работы майнера, твики памяти становятся неактивными в случае проблем с сетью или настроек разгона)
⛏ Gminer v2.22 (добавлена возможность майнить 3Гб картами на алгоритмеr BeamHashIII)
⛏ miniZ v1.6v6 (улучшена стабильность всех алгоритмов, более умная проверка для майнинга ssl / tls, другие мелкие исправления ошибок)
⛏ Обновлен XMRig-CUDA плагин до v6.3.2 (исправлен алгоритм AstroBWT)

Saved searches

Use saved searches to filter your results more quickly

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Pool templates available in Hive 2.0

minershive/hive-pooltemplates

Name already in use

Local
Codespaces

Use Git or checkout with SVN using the web URL.

Work fast with our official CLI. Learn more about the CLI.

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Git stats

Files

Failed to load latest commit information.

README.md

Pool Templates for Hive 2.0

This is a list of pools that are available in Hive for creating Flight Sheet. If you found mistake or the configuration changed please make push request. If you want your pool to be listed here then create yourpool.json and push it, we will review it and add.

Variables

%WAL% — wallet address

%COIN% — wallet coin

%URL% — pool URL address:port

%URL_HOST% — pool address

%URL_PORT% — pool port

%WORKER_NAME% — worker name (from rig name)

%EMAIL% — e-mail for some pool (ex. nanopool)

Miners

astrominer — miner for mining DERO coin on AstroBWTv3 algorithm (CPU)
beamcl — open source BEAM OpenCL miner
beamcuda — open source BEAM CUDA miner
bminer — ethash/tensority/equihash variants/Grin PoW miner
bzminer — ethash/etchash/kawpow/ironfish/radiant miner (AMD/Nvidia/Intel GPUs)
cast-xmr — cryptonight variants (AMD)
ccminer — ccminer & forks (ccminer forks available: alexis, allium, bcd, djm34, enemy, klaust, klaust-yescrypt, nanashi, nevermore, nevermore-x16s, phi-anxmod, rvn, sp-mod, suprminer, suprminer-spmod, tecracoin, tpruvot, vertminer, verus, xaya, xevan, zp)
ckb-miner — CKB (Nervos Network) wallet CPU/OpenCL/CUDA miner
claymore — Claymore’s DUAL ETH GPU AMD&NVidia miner
claymore-x — Claymore’s Cryptonote AMD GPU miner
claymore-z — Claymore’s AMD GPU ZCash miner
cminer — ethash miner for Nvidia
cortex-miner — Cortex CUDA miner
cpuminer-opt — cpuminer-opt CPU miner (forks: JayDDee, cpupower, rkz, rplant, gr)
cryptodredge — multi algo CUDA miner (NVidia)
damominer — multi algo CUDA miner (NVidia)
dero-stratum-miner — AstroBWTv3 miner for CPU
dstm — legacy 0.3.4b and new zhash
eggminergpu — CUDA/OpenCL miner for BIS — Bismuth coin on Eggpool (AMD/Nvidia)
ethminer — ethminer (forks available — ethash, ethercore, firominer, kawpowminer, nsfminer, quarkchain, progpow, teominer, ubiqhash, zilminer)
ewbf — ewbf for equihash and new for equihash algo variants miner
finminer — ethash, randomhash miner (AMD/NVidia/CPU)
gminer — equihash variants CUDA miner
gringoldminer — Cuckaroo29 miner (AMD/NVidia)
grinminer — Cuckaroo29 and Cuckatoo31 miner (AMD/NVidia)
grinprominer — improved version of grinminer (AMD/NVidia)
hellminer — CPU miner for VRSC/VerusCoin
hspminerae — CUDA AE miner (NVidia)
kbminer — AE/Cuckaroo29/Cuckatoo31/VDS miner (AMD/NVidia)
lolminer — equihash variants OpenCL miner (AMD/Nvidia)
miniz — CUDA Equihash variants miner (Nvidia)
nanominer — next generation of finminer (CPU, AMD/NVidia/Intel GPUs)
nbminer — ETH, BTM, Cuckoo miner (NVidia)
nheqminer — fork for CPU mining VerusHash
noncepool-amd — OpenCL miner for BIS — Bismuth coin on Noncepool (AMD)
noncepool-cuda — CUDA miner for BIS — Bismuth coin on Noncepool (Nvidia)
noncerpro-cuda — CUDA miner for NIM — Nimiq coin (Nvidia)
noncerpro-opencl — OpenCL miner for NIM — Nimiq coin (AMD)
noncerpro-kadena — CUDA/OpenCL miner for KDA — Kadena (Nvidia/AMD)
nq-miner — Nimiq GPU OpenCL/CUDA miner (Nvidia/AMD)
onezerominer — Dynex coin miner (Nvidia)
phoenixminer — ethash miner (AMD/NVidia)
rigel — ethash/etchash/kheavyhash/nexapow/sha512256d/blake3 variants for Alephium and Ironfish (Nvidia)
rhminer — randomhash CPU miner (CPU)
sgminer — sgminer forks (avermore, djm34, fancyix, gatelessgate, gm, gm-nicehash, kl, phi, tecracoin)
smine — CKB Spark Miner (AMD)
srbminer-multi — multialgo and multiplatfom miner (CPU, AMD/Nvidia/Intel GPUs)
sushi-miner-cuda — CUDA miner for NIM — Nimiq coin (Nvidia)
sushi-miner-opencl — OpenCL miner for NIM — Nimiq coin (AMD)
t-rex — T-Rex multi algo CUDA miner (NVidia)
teamblackminer — CUDA/OpenCL miner for mining Ethereum, Ethereum Classic and Zilliqa
teamredminer — lyra2z/lyra2v3/phi2/cryptonight-r/v7/v8/half/double/rwz/trtl/x16r/x16rv2 OpenCL miner (AMD, FPGA)
tt-miner — Ethash/Ubqhash/ProgPoW with variants/TEthashV1/MTP/Lyra2rev3 CUDA miner
verthashminer — open-source Verthash CUDA/OpenCL miner
violetminer — CUDA chuckwa/chukwav2 miner
wildrig-multi — multi-algo OpenCL miner (AMD)
xmr-stak — XMR-Stak (AMD,NVidia,CPU cryptonight variants algo with forks arto, alloy, b2n, mox, marketcash, randomx, uplexa)
xmrig — XMRig (CPU cryptonight variants miner with forks: bigbangcore, xmrigcc, hycon, xlarig)
xmrig-new — XMRig (unified) CPU/OCL/CUDA miner for Argon2/RandomX/Cryptonight based algos (available forks: epic, xmrig, mo, randomsfx, xlarig, xdag)
xmrig-amd — XMRig (AMD cryptonight variants miner with forks xmrigcc, hycon)
xmrig-nvidia — XMRig (NVidia cryptonight variants miner with forks fruityminer, hycon)
xpmminer — XPMclient (XPM/Primecoin miner by eXtremal-ik7 for OpenCL and CUDA)
zjazz-cuda — CUDA bitcash/cuckoo/x22i miner (NVidia)
custom — Custom miner package

Other

fah — Folding@Home client

Pool template example

WARNING! Comments in this example only for helping purposes! Don’t use it in pool template file

SSL ports

If your pool contains SSL ports on the same domains then you can add special section «ssl_urls» in geo element

Miners definitions for Hive 2.0

Each file in miners directory contains definitions for corresponding miner.

Below is a description of available options. All fields are optional and may contain null value.

Field name	Type	Default	Description
full_name	string	Display name.
for_amd	boolean	false	Is suitable for AMD GPUs.
for_nvidia	boolean	false	Is suitable for Nvidia GPUs.
for_cpu	boolean	false	Is suitable for CPUs.
for_asic	boolean	false	Is suitable for ASICs.
dual	boolean	false	Supports dual mining.
default_algo	string	Algorithm to use if not reported by miner. This is useful for single-algo miners.
default_fork	string	Fork to use if not configured in flight sheet.
default_units	string	«khs»	Default hashrate units if not reported.
ssl_prefix	string	.
algos	string[] or object	Supported algorithms list. This can be either a simple array of strings or an object where keys are algos and values are display names.
dalgos	string[] or object	Supported algorithms list for dual mining. This can be either a simple array of strings or an object where keys are algos and values are display names.
dual_modes	Map	Allowed algo combinations for dual mining. This is a hashmap where keys are primary algos and values are arrays of allowed secondary algos. Omitting this field means no restrictions at all. If main algo is not present in this field — no dual algos are allowed. If an empty array is set for main algo — no restrictions for this algo.
forks	string[] or object	Available forks list. This can be either a simple array of strings or an object where keys are forks and values are display names.
algomap	object	Algorithms matching. Keys are miner’s algos and values are Hive’s algos.
fork_algo	object	Algorithms for forks. Keys are fork names, values are algos.

Using miners prototypes

To simplify, reducing JSON size and centrally add new miners to mine a particular coin, you can use «miners prototypes» For example: for adding all miners which supports kawpow algorithm vs adding all miners one-by-one it’s can be done by addind only one string «_prototype»: «miners_kawpow» as described below:

If some miner need to override some miner option it can be done

Available such prototypes:

Prototype	Algorithm	Notes
miners_alephium	alephium
miners_astrobwt	astrobwt	V3 implementation
miners_autolykos2	autolykos2
miners_autolykos2_slash_email	autolykos2_slash_email	Nanopool format «WALLET.WORKER/EMAIL»
miners_beamhash	beamhash
miners_cryptonight_gpu	cryptonight_gpu
miners_cryptonight_r	cryptonight_r
miners_cryptonight_v8	cryptonight_v8
miners_cryptonight_xhv	cryptonight_xhv
miners_dynexsolve	dynexsolve
miners_etchash	etchash
miners_ethash	ethash
miners_ethash_4g	ethash_4g
miners_ethash_slash_email	ethash_slash_email	Nanopool format «WALLET.WORKER/EMAIL»
miners_firopow	firopow
miners_ghostrider	ghostrider
miners_hmq1725	hmq1725
miners_ironfish	ironfish
miners_kaspa	kaspa
miners_kawpow	kawpow
miners_kawpow_slash_email	kawpow_slash_email	Nanopool format «WALLET.WORKER/EMAIL»
miners_minotaurx	minotaurx
miners_minotaurx_jiimp	minotaurx_jiimp	JIIMP format
miners_nexapow	nexapow
miners_octopus	octopus
miners_olhash	olhash
miners_progpow_sero	progpow_sero
miners_progpow_zano	progpow_zano
miners_radiant	radiant
miners_randomhash2_with_email	randomhash2_with_email	Nanopool format «WALLET.WORKER/EMAIL»
miners_randomx	randomx	Nicehash format «wallet.worker»
miners_randomx_plus	randomx_plus	xmrpool format «wallet+worker»
miners_randomx_slash_email	randomx_slash_email	Nanopool format «WALLET.WORKER/EMAIL»
miners_randomx_worker	randomx_worker	worker name in password
miners_randomx_xdag	randomx_xdag
miners_sha3d	sha3d
miners_ubqhash	ubqhash
miners_verthash	verthash
miners_verushash	verushash
miners_zelhash	zelhash

Changelog for Hive 2.0

Located in file changelog.md .

Each section of the file represents one release and consist of heading, optionally followed by body.

Heading

Starts with at least one # sign and contain definition string in such format:

[ LINUX | ASIC | Windows ] [ IMAGE RELEASE ] Version Date YYYY-MM-DD

Contains any text, mardown syntax is supported.

All lines until next heading are considered as body, empty leading and trailing lines are skipped.

SSL warning in hive

When I try to run hive commands,I get the following warning alongwith the output.

Establishing SSL connection without server’s identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn’t set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to ‘false’. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

I tried to edit hive-site.xml jdbc url as

But still I’m getting this warning. What is the proper syntax to disable ssl? Thanks in advance.

SSL configuration in Hive

Right after the installation of the Hive Platform WebUI is available only on HTTP.

You can configure SSL by one of the following methods:

upload your SSL certificates to the Platform and configure HTTP/HTTPs;
configure HTTP or TCP Reverse proxy with SSL termination.

Upload SSL certificates to Hive

Upload SSL certificates in the PEM format to the directory /opt/hw-bw/ssl . Please note that uploaded files will not be overwritten when upgrading to the next version, so feel free to store them in this directory.

You can change default directory if it does not suit you:

Open the file /opt/hw-bw/config/user.ini (root privileges required);

Add the b.ssl.dir option to the [main] section and indicate new path:

To apply changes, run the command:

Configure HTTP/HTTPs

To configure HTTP to HTTPs redirect add the following properties to the [main] section of /opt/hw-bw/config/user.ini file (root privileges required):

b.ssl.enabled — option, which enables SSL. Possible values:

no_ssl — only insecure HTTP-connection is used (default value);
ssl_both — both insecure HTTP-connection and secure HTTPs-connection are used;
ssl_redirect — redirection from insecure connection HTTP-connection to secure HTTPs-connection;
ssl_only — only secure HTTPs-connection is used.

b.deck.ip.expose — IP address for insecure HTTP-connection ( 0.0.0.0 — public IP address, 127.0.0.1 — local IP address). Required for no_ssl , ssl_both and ssl_redirect .

To apply changes, run the command:

Configure HTTP or TCP Reverse proxy with SSL termination

If necessary, you can run Hive behind your own reverse proxy. For example, configure nginx , which will proxy all requests for Hive.

To use your proxy server on the same machine, change the following parameters in /opt/hw-bw/config/user.ini :

To apply changes, run the command (root privileges required):

Example of nginx configuration as HTTP Reverse proxy

Example of nginx configuration:

10001 — port number you set to b.deck.port.expose in user.ini file;
yourhive.example.com — hostname or IP address of your virtual machine.

Example of Let’s Encrypt with certbot configuration. You should set this configuration before you issue certificates using certbot .

10001 — port number you set to b.deck.port.expose in user.ini file;
yourhive.example.com — hostname or IP address of your virtual machine.

Convert CA-certificate from .PFX

You may need to convert CA from .pfx if: * it is necessary to transfer issues between Hive and Apiary via secure channel; * it is impossible to use global valid certificates; * it is necessary to use local Certificate Authority to issue certificates.

Create SSL certificate:

Note: remove the -out option from the commands and the key with other information will be displayed on the screen. In this case, copy everything from the lines BEGIN PRIVATE KEY / BEGIN CERTIFICATE to END PRIVATE KEY / END CERTIFICATE and save it to a file.

Create a root certificate:

Note: you will get a chain of root certificates. You will need only last one certificate in this chain. For example, you can open file for editing and delete everything except the last one.

Concatenate files into one certificate:

Copy SSL certificates to a virtual machine with installed Platform to a directory with SSL certificates:

Похожие публикации:

Rarible или opensea что лучше
За сколько я накоплю калькулятор
Как узнать баланс транспортной карты через сбербанк онлайн
Что будет с курсом доллара к рублю

How do I set up an SSL connection between Hive on Amazon EMR and a metastore on Amazon RDS for MySQL?

I want to set up an SSL connection between Apache Hive and a metastore on an Amazon Relational Database Service (Amazon RDS) MySQL DB instance. How can I do that?

Short description

Set up an encrypted connection between Hive and an external metastore using an SSL certificate. You can set up this connection when you launch a new Amazon EMR cluster or after the cluster is running.

Resolution

Note: The following steps are tested with Amazon EMR release version 5.36.0 and Amazon RDS for MySQL version 8.0.28

Set up the SSL connection on a new Amazon EMR cluster

1. Run a command similar to the following to create an Amazon RDS for MySQL DB instance. Replace $RDS_LEADER_USER_NAME, $RDS_LEADER_PASSWORD, $RDS_VPC_SG, and $DB_SUBNET_GROUP with your user name, password, security group, and DB Subnet Group respectively.

For more information, see create-db-instance.

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, be sure that you’re using the most recent version of the CLI.

aws rds create-db-instance --db-name hive --db-instance-identifier mysql-hive-meta --db-instance-class db.t2.micro --engine mysql --engine-version 8.0.28 --db-subnet-group-name $DB_SUBNET_GROUP --master-username $RDS_LEADER_USER_NAME --master-user-password $RDS_LEADER_PASSWORD --allocated-storage 20 --storage-type gp2 --vpc-security-group-ids $RDS_VPC_SG --publicly-accessible

2. Connect to the Amazon RDS for MySQL DB instance as the primary user. Then, create a user for the Hive metastore, as shown in the following example.

Important: Be sure that you restrict access for this user to the DB instance that you created in Step 1.

mysql -h mysql-hive-meta.########.us-east-1.rds.amazonaws.com -P 3306 -u $RDS_LEADER_USER_NAME -p Enter password: $RDS_LEADER_PASSWORD CREATE USER 'hive_user'@'%' IDENTIFIED BY 'hive_user_password' REQUIRE SSL; REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive_user'@'%'; GRANT ALL PRIVILEGES ON hive.* TO 'hive_user'@'%'; FLUSH PRIVILEGES;

3. Create a JSON configuration file similar to the following. Replace hive_user and hive_user_password with the values that you used in the JSON script in Step 2. Replace the endpoint in the JDBC URL with the endpoint for your RDS DB instance.

You use this file to launch the Amazon EMR cluster in the next step. The file enables Hive makes an SSL connection to the RDS DB instance. For more information, see Using new SSL/TLS certificates for MySQL DB instances.

4. In the security group that's associated with the Amazon RDS for MySQL instance, create an inbound rule with the following parameters:
For Type, choose MYSQL/Aurora (3306).
For Protocol, TCP (6) is selected by default.
For Port Range, 3306 is selected by default.
For Source, enter the Group ID of the Amazon EMR-managed security group that's associated with the leader node.

This rule allows the Amazon EMR cluster's leader node to access the Amazon RDS instance. For more information, see Overview of VPC security groups.

5. Run the create-cluster command to launch an Amazon EMR cluster using the JSON file from Step 3, along with a bootstrap action. The bootstrap action downloads the SSL certificate to /home/hadoop/ on the leader node.

aws emr create-cluster --applications Name=Hadoop Name=Hive --tags Name="EMR Hive Metastore SSL" --ec2-attributes KeyName=$EC2_KEY_PAIR,InstanceProfile=EMR_EC2_DefaultRole,SubnetId=$EMR_SUBNET,EmrManagedSlaveSecurityGroup=$EMR_CORE_AND_TASK_VPC_SG,EmrManagedMasterSecurityGroup=$EMR_MASTER_VPC_SG --service-role EMR_DefaultRole --release-label emr-5.36.0 --log-uri $LOG_URI --name "Hive External Metastore RDS MySQL w/ SSL" --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m4.xlarge,Name="Master- 1" --configurations file:////hive-ext-meta-mysql-ssl.json --bootstrap-actions Path=s3://elasticmapreduce/bootstrap-actions/run-if,Args=["instance.isMaster=true","cd /home/hadoop && wget -S -T 10 -t 5 https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem"]

7. Open a hive session on the leader node. Then, create any table (to be used for testing purposes).

hive> create table tb_test (col1 STRING, col2 BIGINT); OK Time taken: 2.371 seconds hive> describe tb_test; OK col1 string col2 bigint Time taken: 0.254 seconds, Fetched: 2 row(s)

8. Connect to the Amazon RDS for MySQL metastore using the mysql client on the leader node. Then, verify the table metadata in the metastore. If the metadata corresponds to the table that you created in the previous step exists, then the SSL connection is working.

mysql -h mysql-hive-meta.########.us-east-1.rds.amazonaws.com -P 3306 -u $RDS_LEADER_USER_NAME -p Enter password: $RDS_LEADER_PASSWORD mysql> use hive; Database changed mysql> select t1.OWNER, t1.TBL_NAME, t1.TBL_TYPE, s1.INPUT_FORMAT, s1.OUTPUT_FORMAT, s1.LOCATION from TBLS t1 inner join SDS s1 on s1.SD_ID = t1.SD_ID where t1.TBL_NAME = 'tb_test'\G *************************** 1. row *************************** OWNER: hadoop TBL_NAME: tb_test TBL_TYPE: MANAGED_TABLE INPUT_FORMAT: org.apache.hadoop.mapred.TextInputFormat OUTPUT_FORMAT: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat LOCATION: hdfs://ip-xxx-xx-xx-xxx.ec2.internal:8020/user/hive/warehouse/tb_test 1 row in set (0.23 sec) mysql> select t1.OWNER, t1.TBL_NAME, c1.COLUMN_NAME, c1.TYPE_NAME from TBLS t1 inner join SDS s1 on s1.SD_ID = t1.SD_ID inner join COLUMNS_V2 c1 on c1.CD_ID = s1.CD_ID where t1.TBL_NAME = 'tb_test'; +--------+----------+-------------+-----------+ | OWNER | TBL_NAME | COLUMN_NAME | TYPE_NAME | +--------+----------+-------------+-----------+ | hadoop | tb_test | col1 | string | | hadoop | tb_test | col2 | bigint | +--------+----------+-------------+-----------+ 2 rows in set (0.22 sec)

Set up the SSL connection on a running Amazon EMR cluster

Note: The following steps assume that you have an Amazon RDS for MySQL DB instance.

2. Run the following command to download the SSL certificate to /home/hadoop/ on the leader node:

cd /home/hadoop && wget -S -T 10 -t 5 https://s3.amazonaws.com/rds-downloads/global-bundle.pem

3. In the /etc/hive/conf.dist directory, add or edit the following lines in the hive-site.xml file:

 javax.jdo.option.ConnectionURL jdbc:mysql://mysql-hive-meta.########.us-east-1.rds.amazonaws.com:3306/hive?createDatabaseIfNotExist=true&useSSL=true&serverSslCert=/home/hadoop/global-bundle.pem JDBC URL for the metastore database  javax.jdo.option.ConnectionUserName hive_user User name for the metastore database  javax.jdo.option.ConnectionPassword HIVE_USER_PASSWORD Password for metastore database

This syntax allows an SSL connection to the RDS DB instance. Make sure to replace the endpoint in the JDBC URL with the endpoint for your RDS DB instance.

Important: The ampersand (&) is a special character in XML. To use an ampersand in hive-site.xml, such as in the JDBC string, you must use "&" instead of "&." Otherwise, you get an error when you restart hive-hcatalog-server.

4. Run a command similar to the following to test the SSL connection:

mysql -h mysql-hive-meta.########.us-east-1.rds.amazonaws.com -P 3306 -u hive_user -p --ssl-ca /home/hadoop/rds-combined-ca-bundle.pem

5. Restart hive-hcatalog-server on the leader node. For more information, see Stopping and restarting processes.

6. Verify that the services restarted successfully:

sudo systemctl status hive-hcatalog-server.service

7. Open a hive session on the leader node. Then, create any table (to be used for testing purposes).

hive> create table tb_test (col1 STRING, col2 BIGINT); OK Time taken: 2.371 seconds hive> describe tb_test; OK col1 string col2 bigint Time taken: 0.254 seconds, Fetched: 2 row(s)

$ mysql -h mysql-hive-meta.########.us-east-1.rds.amazonaws.com -P 3306 -u $RDS_LEADER_USER_NAME -p Enter password: $RDS_LEADER_PASSWORD mysql> use hive; Database changed mysql> select t1.OWNER, t1.TBL_NAME, t1.TBL_TYPE, s1.INPUT_FORMAT, s1.OUTPUT_FORMAT, s1.LOCATION from TBLS t1 inner join SDS s1 on s1.SD_ID = t1.SD_ID where t1.TBL_NAME = 'tb_test'\G *************************** 1. row *************************** OWNER: hadoop TBL_NAME: tb_test TBL_TYPE: MANAGED_TABLE INPUT_FORMAT: org.apache.hadoop.mapred.TextInputFormat OUTPUT_FORMAT: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat LOCATION: hdfs://ip-xxx-xx-xx-xxx.ec2.internal:8020/user/hive/warehouse/tb_test 1 row in set (0.23 sec) mysql> select t1.OWNER, t1.TBL_NAME, c1.COLUMN_NAME, c1.TYPE_NAME from TBLS t1 inner join SDS s1 on s1.SD_ID = t1.SD_ID inner join COLUMNS_V2 c1 on c1.CD_ID = s1.CD_ID where t1.TBL_NAME = 'tb_test'; +--------+----------+-------------+-----------+ | OWNER | TBL_NAME | COLUMN_NAME | TYPE_NAME | +--------+----------+-------------+-----------+ | hadoop | tb_test | col1 | string | | hadoop | tb_test | col2 | bigint | +--------+----------+-------------+-----------+ 2 rows in set (0.22 sec)

Troubleshoot hive-hcatalog-server restart errors

You might get an error message similar to the following when you try to restart hive-hcatalog-server:

2020-08-20T14:18:50,750 WARN [main] org.apache.hadoop.hive.metastore.HiveMetaStore - Retrying creating default database after error: Unable to open a test connection to the given database. JDBC url = jdbc:mysql://mysql-hive-meta.########.us-east-1.rds.amazonaws.com:3306/hive?createDatabaseIfNotExist=true&useSSL=true&serverSSlCert=/home/hadoop/global-bundle.pem, username = masteruser. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------ java.sql.SQLException: Host '172.31.41.187' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'

This error message typically occurs when the Amazon RDS for MySQL DB instance blocks the Amazon EMR cluster's leader node as a security precaution.

To resolve this error, connect to a different local machine or Amazon Elastic Compute Cloud (Amazon EC2) instance that has the mysqladmin tool installed. Run the following command to flush the leader node from the DB instance.

mysqladmin -h mysql-hive-meta.########.us-east-1.rds.amazonaws.com -P 3306 -u $RDS_LEADER_USER_NAME -p flush-hosts Enter password: $RDS_LEADER_PASSWORD

Then, restart hive-hcatalog-server.