posted by 떠돌이늑대 2020. 7. 20. 14:39

binlog_row_metadata

  • Description: Controls the format used for binlog metadata logging.
    • NO_LOG: No metadata is logged (default).
    • MINIMAL: Only metadata required by a slave is logged.
    • FULL: All metadata is logged.
  • Commandline: --binlog-row-metadata=value
  • Scope: Global, Session
  • Dynamic: Yes
  • Data Type: enum
  • Default Value: NO_LOG
  • Valid Values: NO_LOG, MINIMAL, FULL
  • Introduced: MariaDB 10.5.0

 

innodb_instant_alter_column_allowed

  • Description:
    • If a table is altered using ALGORITHM=INSTANT, it can force the table to use a non-canonical format: A hidden metadata record at the start of the clustered index is used to store each column's DEFAULT value. This makes it possible to add new columns that have default values without rebuilding the table. Starting with MariaDB 10.4, a BLOB in the hidden metadata record is used to store column mappings. This makes it possible to drop or reorder columns without rebuilding the table. This also makes it possible to add columns to any position or drop columns from any position in the table without rebuilding the table. If a column is dropped without rebuilding the table, old records will contain garbage in that column's former position, and new records will be written with NULL values, empty strings, or dummy values.
    • This is generally not a problem. However, there may be cases where you want to avoid putting a table into this format. For example, to ensure that future UPDATE operations after an ADD COLUMN will be performed in-place, to reduce write amplification. (Instantly added columns are essentially always variable-length.) Also avoid bugs similar to MDEV-19916, or to be able to export tables to older versions of the server.
    • This variable has been introduced as a result, with the following values:
    • never (0): Do not allow instant add/drop/reorder, to maintain format compatibility with MariaDB 10.x and MySQL 5.x. If the table (or partition) is not in the canonical format, then any ALTER TABLE (even one that does not involve instant column operations) will force a table rebuild.
    • add_last (1, default in 10.3): Store a hidden metadata record that allows columns to be appended to the table instantly (MDEV-11369). In 10.4 or later, if the table (or partition) is not in this format, then any ALTER TABLE (even one that does not involve column changes) will force a table rebuild.
    • add_drop_reorder (2, default): From MariaDB 10.4 only. Like 'add_last', but allow the metadata record to store a column map, to support instant add/drop/reorder of columns.
  • Commandline: --innodb-instant-alter-column-allowed=value
  • Scope: Global
  • Dynamic: Yes
  • Data Type: enum
  • Valid Values:
  • Default Value:
  • Introduced: MariaDB 10.3.23, MariaDB 10.4.13, MariaDB 10.5.3

 

 

performance_schema_events_transactions_history_long_size

  • Description: Number of rows in events_transactions_history_long table. Use 0 to disable, -1 for automated sizing.
  • Commandline: --performance-schema-events-transactions-history-long-size=#
  • Scope: Global
  • Dynamic: No
  • Data Type: numeric
  • Default Value: -1
  • Range: -1 to 1048576
  • Introduced: MariaDB 10.5.2

 

 

performance_schema_events_transactions_history_size

  • Description:Number of rows per thread in events_transactions_history. Use 0 to disable, -1 for automated sizing.
  • Commandline: --performance-schema-events-transactions-history-size=#
  • Scope: Global
  • Dynamic: No
  • Data Type: numeric
  • Default Value: -1
  • Range: -1 to 1024
  • Introduced: MariaDB 10.5.2

 

 

performance_schema_max_index_stat

  • Description: Maximum number of index statistics for instrumented tables. Use 0 to disable, -1 for automated scaling.
  • Commandline: --performance-schema-max-index-stat=#
  • Scope: Global
  • Dynamic: No
  • Data Type: numeric
  • Default Value: -1
  • Range: -1 to 1048576
  • Introduced: MariaDB 10.5.2

 

 

performance_schema_max_memory_classes

  • Description: Maximum number of memory pool instruments.
  • Commandline: --performance-schema-max-memory-classes=#
  • Scope: Global
  • Dynamic: No
  • Data Type: numeric
  • Default Value: 320
  • Range: 0 to 1024
  • Introduced: MariaDB 10.5.2

 

 

performance_schema_max_metadata_locks

  • Description: Maximum number of metadata locks. Use 0 to disable, -1 for automated scaling.
  • Commandline: --performance-schema-max-metadata-locks=#
  • Scope: Global
  • Dynamic: No
  • Data Type: numeric
  • Default Value: -1
  • Range: -1 to 104857600
  • Introduced: MariaDB 10.5.2

 

 

 

performance_schema_max_prepared_statement_instances

  • Description: Maximum number of instrumented prepared statements. Use 0 to disable, -1 for automated scaling.
  • Commandline: --performance-schema-max-prepared-statement-instances=#
  • Scope: Global
  • Dynamic: No
  • Data Type: numeric
  • Default Value: -1
  • Range: -1 to 1048576
  • Introduced: MariaDB 10.5.2

 

 

 

performance_schema_max_program_instances

  • Description: Maximum number of instrumented programs. Use 0 to disable, -1 for automated scaling.
  • Commandline: --performance-schema-max-program-instances=#
  • Scope: Global
  • Dynamic: No
  • Data Type: numeric
  • Default Value: -1
  • Range: -1 to 1048576
  • Introduced: MariaDB 10.5.2

 

 

performance_schema_max_sql_text_length

  • Description: Maximum length of displayed sql text.
  • Commandline: --performance-schema-max-sql-text-length=#
  • Scope: Global
  • Dynamic: No
  • Data Type: numeric
  • Default Value: 1024
  • Range: 0 to 1048576
  • Introduced: MariaDB 10.5.2

 

 

performance_schema_max_statement_stack

  • Description: Number of rows per thread in EVENTS_STATEMENTS_CURRENT.
  • Commandline: --performance-schema-max-statement-stack=#
  • Scope: Global
  • Dynamic: No
  • Data Type: numeric
  • Default Value: 10
  • Range: 1 to 256
  • Introduced: MariaDB 10.5.2

 

 

performance_schema_max_table_lock_stat

  • Description: Maximum number of lock statistics for instrumented tables. Use 0 to disable, -1 for automated scaling.
  • Commandline: --performance-schema-max-table-lock-stat=#
  • Scope: Global
  • Dynamic: No
  • Data Type: numeric
  • Default Value: -1
  • Range: -1 to 1048576
  • Introduced: MariaDB 10.5.2

 

 

s3_access_key

  • Description: The AWS access key to access your data. See mysqld startup options for S3.
  • Commandline: --s3-access-key=val
  • Scope: Global
  • Dynamic: No
  • Data Type: String
  • Default Value: (Empty)
  • Introduced: MariaDB 10.5.4

 

 

s3_block_size

  • Description: The default block size for a table, if not specified in CREATE TABLE. Set to 4M as default. See mysqld startup options for S3.
  • Commandline: --s3-block-size=#
  • Scope: Global
  • Dynamic: Yes
  • Data Type: Numeric
  • Default Value: 4194304
  • Range: 4194304 to 16777216
  • Introduced: MariaDB 10.5.4

s3_bucket

  • Description: The AWS bucket where your data should be stored. All MariaDB table data is stored in this bucket. See mysqld startup options for S3.
  • Commandline: --s3-bucket=val
  • Scope: Global
  • Dynamic: No
  • Data Type: String
  • Default Value: MariaDB
  • Introduced: MariaDB 10.5.4

s3_debug

  • Description: Generates a trace file from libmarias3 on stderr (mysqld.err) for debugging the S3 protocol.
  • Commandline: --s3-debug{=0|1}
  • Scope: Global
  • Dynamic: No
  • Data Type: Boolean
  • Valid Values: 0 or 1
  • Default Value: 0
  • Introduced: MariaDB 10.5.4

s3_host_name

  • Description: Hostname for the S3 service. "s3.amazonaws.com", Amazon S3 service, by default
  • Commandline: --s3-host-name=val
  • Scope: Globa;
  • Dynamic: No
  • Data Type: String
  • Default Value: s3.amazonaws.com
  • Introduced: MariaDB 10.5.4

s3_pagecache_age_threshold

  • Description: This characterizes the number of hits a hot block has to be untouched until it is considered aged enough to be downgraded to a warm block. This specifies the percentage ratio of that number of hits to the total number of blocks in the page cache.
  • Commandline: --s3-pagecache-age-threshold=val
  • Scope: Global
  • Dynamic: Yes
  • Data Type: Numeric
  • Default Value: 300
  • Range: 100 to 18446744073709551615
  • Introduced: MariaDB 10.5.4

s3_pagecache_buffer_size

  • Description: The size of the buffer used for index blocks for S3 tables. Increase this to get better index handling (for all reads and multiple writes) to as much as you can afford. Size can be adjusted in blocks of 8192.
  • Commandline: --s3-pagecache-buffer-size=val
  • Scope: Global
  • Dynamic: No
  • Data Type: Numeric
  • Default Value: 134217728 (128M)
  • Range: 33554432 to 18446744073709551615
  • Introduced: MariaDB 10.5.4

s3_pagecache_division_limit

  • Description: The minimum percentage of warm blocks in key cache.
  • Commandline: --s3-pagecache-division-limit=val
  • Scope: Global
  • Dynamic: Yes
  • Data Type: Numeric
  • Default Value: 100
  • Range: 1 to 100
  • Introduced: MariaDB 10.5.4

s3_pagecache_file_hash_size

  • Description: Number of hash buckets for open files. Default 512. If you have a lot of S3 files open you should increase this for faster flush of changes. A good value is probably 1/10 of number of possible open S3 files.
  • Commandline: --s3-pagecache-file-hash-size=#
  • Scope: Global
  • Dynamic: No
  • Data Type: Numeric
  • Default Value: 512
  • Range: 32 to 16384
  • Introduced: MariaDB 10.5.4

s3_protocol_version

  • Description: Protocol used to communication with S3. One of "Auto", "Amazon" or "Original" where "Auto" is the default. If you get errors like "8 Access Denied" when you are connecting to another service provider, then try to change this option. The reason for this variable is that Amazon has changed some parts of the S3 protocol since they originally introduced it but other service providers are still using the original protocol.
  • Commandline: --s3-protocol-version=val
  • Scope: Global
  • Dynamic: Yes
  • Data Type: Enum
  • Valid Values: Auto, Amazon or Original
  • Default Value: Auto
  • Introduced: MariaDB 10.5.4

s3_region

  • Description: The AWS region where your data should be stored. See mysqld startup options for S3.
  • Commandline: --s3-region=val
  • Scope: Global
  • Dynamic: No
  • Data Type: String
  • Default Value: (Empty)
  • Introduced: MariaDB 10.5.4

s3_replicate_alter_as_create_select

  • Description: When converting S3 table to local table, log all rows in binary log. This allows the slave to replicate CREATE TABLE .. SELECT FROM s3_table even it the slave doesn't have access to the original s3_table.
  • Commandline: --s3-replicate-alter-as-create-select{=0|1}
  • Scope: Global
  • Dynamic: No
  • Data Type: Boolean
  • Default Value: 1
  • Introduced: MariaDB 10.5.4

s3_secret_key

  • Description: The AWS secret key to access your data. See mysqld startup options for S3.
  • Commandline: --s3-secret-key=val
  • Scope: Global
  • Dynamic: No
  • Data Type: String
  • Default Value: (Empty)
  • Introduced: MariaDB 10.5.4

s3_slave_ignore_updates

  • Description: Should be set if master and slave share the same S3 instance. This tells the slave that it can ignore any updates to the S3 tables as they are already applied on the master.
  • Commandline: --s3-slave-ignore-updates{=0|1}
  • Scope: Global
  • Dynamic: No
  • Data Type: Boolean
  • Default Value: 0
  • Introduced: MariaDB 10.5.4

sql_if_exists

  • Description: If set to 1, adds an implicit IF EXISTS to ALTER, RENAME and DROP of TABLES, VIEWS, FUNCTIONS and PACKAGES. This variable is mainly used in replication to tag DDLs that can be ignored on the slave if the target table doesn't exist.
  • Commandline: --sql-if-exists[={0|1}]
  • Scope: Global, Session
  • Dynamic: Yes
  • Data Type: boolean
  • Default Value: OFF
  • Introduced: MariaDB 10.5.2

 

 

thread_pool_dedicated_listener

  • Description: If set to 1, then each group will have its own dedicated listener, and the listener thread will not pick up work items. As a result, the queueing time in the Information Schema Threadpool_Queues and the actual queue size in the Information Schema Threadpool_Groups table will be more exact, since IO requests are immediately dequeued from poll, without delay.
    • This system variable is only meaningful on Unix.
  • Commandline: thread-pool-dedicated-listener={0|1}
  • Scope:
  • Dynamic:
  • Data Type: boolean
  • Default Value: 0
  • Introduced: MariaDB 10.5.0

 

 

thread_pool_exact_stats

  • Description: If set to 1, provides better queueing time statistics by using a high precision timestamp, at a small performance cost, for the time when the connection was added to the queue. This timestamp helps calculate the queuing time shown in the Information Schema Threadpool_Queues table.
    • This system variable is only meaningful on Unix.
  • Commandline: thread-pool-exact-stats={0|1}
  • Scope:
  • Dynamic:
  • Data Type: boolean
  • Default Value: 0
  • Introduced: MariaDB 10.5.0

'컴이야기 > MariaDB' 카테고리의 다른 글

MaxScale HA setup using Keepalived and MaxCtrl  (0) 2021.01.07
MariaDB installing in minimal install RHEL8  (0) 2020.07.20
Maxscale  (0) 2020.06.30
CRUD  (0) 2020.06.12
Using Galera Replication to Create Geo-distributed Clusters  (0) 2020.06.12
posted by 떠돌이늑대 2020. 7. 20. 13:03

mysql_secure_installation  error

 

'컴이야기 > MariaDB' 카테고리의 다른 글

MaxScale HA setup using Keepalived and MaxCtrl  (0) 2021.01.07
10.5 에서 추가된 시스템 변수  (0) 2020.07.20
Maxscale  (0) 2020.06.30
CRUD  (0) 2020.06.12
Using Galera Replication to Create Geo-distributed Clusters  (0) 2020.06.12
posted by 떠돌이늑대 2020. 6. 12. 13:24

Create,

Read,

Update,

Delete

'컴이야기 > MariaDB' 카테고리의 다른 글

MariaDB installing in minimal install RHEL8  (0) 2020.07.20
Maxscale  (0) 2020.06.30
Using Galera Replication to Create Geo-distributed Clusters  (0) 2020.06.12
List of Galera Cluster Status variables  (0) 2020.06.08
sysbench Test  (0) 2020.06.02
posted by 떠돌이늑대 2020. 6. 12. 11:13

https://youtu.be/-UxNUKYh7Vw

 "wsrep_provider_options" with "gmcast.segment= "

 

'컴이야기 > MariaDB' 카테고리의 다른 글

Maxscale  (0) 2020.06.30
CRUD  (0) 2020.06.12
List of Galera Cluster Status variables  (0) 2020.06.08
sysbench Test  (0) 2020.06.02
Maxscale Fail-Over, Switch-Over, Re-Join  (0) 2020.05.26
posted by 떠돌이늑대 2020. 6. 8. 11:20

https://mariadb.com/kb/en/galera-cluster-status-variables

 

Galera Cluster Status Variables

Galera Cluster Status Variables

mariadb.com

wsrep_applier_thread_count
설명 :이 유형의 슬레이브 스레드 수를 명확하게하기 위해 현재의 적용자 스레드 수를 저장합니다.

 

wsrep_apply_oooe
설명 : 병렬화 효율성의 지표 인 쓰기 세트가 순서대로 적용되지 않은 빈도입니다.

 

wsrep_apply_oool
설명 : 시퀀스 번호가 낮은 쓰기 세트를 시퀀스 번호가 낮은 쓰기 세트보다 먼저 적용하여 쓰기 세트가 느리다는 것을 나타냅니다.

 

wsrep_apply_window
설명 : 가장 높고 가장 낮은 동시 적용 seqno 사이의 평균 거리.

 

wsrep_cert_deps_distance
설명 : 병렬로 적용 할 수있는 가장 높은 시퀀스 번호와 가장 낮은 시퀀스 번호 사이의 평균 거리 또는 가능한 병렬화 수준.

 

....

'컴이야기 > MariaDB' 카테고리의 다른 글

CRUD  (0) 2020.06.12
Using Galera Replication to Create Geo-distributed Clusters  (0) 2020.06.12
sysbench Test  (0) 2020.06.02
Maxscale Fail-Over, Switch-Over, Re-Join  (0) 2020.05.26
Maxscale serversize ,weightby  (0) 2020.05.22
posted by 떠돌이늑대 2020. 6. 2. 15:03

CPU 코어 1개

[root@localhost ~]# sysbench --mysql-db=cfpc --mysql-host=192.168.1.18 --mysql-port=4006 --mysql-user=maxscale --mysql-password=maxscale123 --max-time=600 --threads=40 /usr/share/sysbench/oltp_insert.lua run
WARNING: --max-time is deprecated, use --time instead
sysbench 1.0.20 (using bundled LuaJIT 2.1.0-beta2)

Running the test with following options:
Number of threads: 40
Initializing random number generator from current time


Initializing worker threads...

Threads started!

SQL statistics:
queries performed:
read: 0
write: 6085278
other: 0
total: 6085278
transactions: 6085278 (10141.57 per sec.)
queries: 6085278 (10141.57 per sec.)
ignored errors: 0 (0.00 per sec.)
reconnects: 0 (0.00 per sec.)

General statistics:
total time: 600.0326s
total number of events: 6085278

Latency (ms):
min: 0.17
avg: 3.94
max: 662.70
95th percentile: 8.58
sum: 23990186.08

Threads fairness:
events (avg/stddev): 152131.9500/533.58
execution time (avg/stddev): 599.7547/0.01


CPU 코어 4개

[root@localhost ~]# sysbench --mysql-db=cfpc --mysql-host=192.168.1.18 --mysql-port=4006 --mysql-user=maxscale --mysql-password=maxscale123 --max-time=600 --threads=40 /usr/share/sysbench/oltp_insert.lua run
WARNING: --max-time is deprecated, use --time instead
sysbench 1.0.20 (using bundled LuaJIT 2.1.0-beta2)

Running the test with following options:
Number of threads: 40
Initializing random number generator from current time


Initializing worker threads...

Threads started!

SQL statistics:
queries performed:
read: 0
write: 18718049
other: 0
total: 18718049
transactions: 18718049 (31195.96 per sec.)
queries: 18718049 (31195.96 per sec.)
ignored errors: 0 (0.00 per sec.)
reconnects: 0 (0.00 per sec.)

General statistics:
total time: 600.0144s
total number of events: 18718049

Latency (ms):
min: 0.15
avg: 1.28
max: 426.97
95th percentile: 3.02
sum: 23967305.01

Threads fairness:
events (avg/stddev): 467951.2250/18031.93
execution time (avg/stddev): 599.1826/0.03

posted by 떠돌이늑대 2019. 6. 14. 13:03
WSREP_SST: [ERROR] xtrabackup_checkpoints missing, failed innobackupex/SST on donor (20190614 19:30:21.392)
WSREP_SST: [ERROR] Cleanup after exit with status:2 (20190614 19:30:21.394)

 

MariaDB 버전변경하면서 테스트를 하던 중 Galera Cluster를 새로 구성하면서, 다른 2 node 들이 위와 같은 에러가

발생하면서 DB 구동이 실패가 되었었다.

무슨 문제였는지  알수가 없었다.

wsrep_sst_method 옵션을 변경해도 error 가 발생되면서 DB 구동이 실패가 되었었다.

 

첫번째 노드의 data가 들어있는 디렉토리를 살펴보면,

 

innobackup.backup.log 라는 파일이 있다.

 

다른 노드에서 sst를 요청하면 과정과 결과가 기록이 된다.

 

2019-06-14 19:30:21 140471879198592 [ERROR] InnoDB: Tablespaces for test/t1 have been found in two places;
Location 1: SpaceID: 21550 File: ./test/t1.ibd
Location 2: SpaceID: 21550 File: /data/test/test/t1.ibd
You must delete one of them.
190614 19:30:21 [ERROR] mysqld got signal 6

 

이러한 기록들이 있었다.

 

저 테이블에 문제가 있었나보다.

 

실제로 첫번째 node 의 데이터를 mysqldump 로 백업하려했더니 저 테이블때문에 계속 실패했다. 다른 한 테이블도

그래서 제외시켜서 백업했더니 잘 되었었다.

 

mysqldump -uroot -predhat --databases test --ignore-table=test.t1 --ignore-table=test.t2 > /mysqldump/test.sql

결국 첫번째 노드에서 문제있었던 두 테이블을 삭제하니

 

Galera Cluster도 정상으로 되었다.

 

 

 

 

'컴이야기 > MariaDB' 카테고리의 다른 글

wsrep_sst_method with IPv6  (0) 2019.10.21
Maxscale 2.3.6  (1) 2019.07.12
Galera Cluster Replication.  (0) 2019.03.18
maxscale 을 사용해보자. [진행중]  (0) 2018.11.07
Galera cluster 설치 삽질과정 (완료)  (0) 2018.11.02
posted by 떠돌이늑대 2019. 3. 18. 15:01

2개의 Galera Cluster를 만들고, 각 클러스터의 한 노드끼리 서로간의 Replication 설정을 했다.

 

그러나 왠걸 Replication  설정을 한 노드들끼리의 복제는 되지만, 다른 노드에서 실행하면 클러스터 내부에서만 복제가 되고, 다른 클러스터로는 복제가 되질 않았다.

 

무엇일까... 찾아보게 되었고,

 

서로 Replication을 맺어준 노드에서 my.cnf 에

 

이 옵션값을 선언해 주어야지만 다른 노드에서의 명령도 전달복제가 되었다.

 

log_slave_updates 의 값을 선언을 해주어야 한다.

 

log_slave_updates

  • Description: If set to 0, the default, updates on a slave received from a master during replication are not logged in the slave's binary log. If set to 1, they are. The slave's binary log needs to be enabled for this to have an effect. Set to 1 if you want to daisy-chain the slaves.
  • Commandline: --log-slave-updates
  • Scope: Global
  • Dynamic: No
  • Data Type: boolean
  • Default Value: OFF

'컴이야기 > MariaDB' 카테고리의 다른 글

Maxscale 2.3.6  (1) 2019.07.12
Galera Cluster error  (0) 2019.06.14
maxscale 을 사용해보자. [진행중]  (0) 2018.11.07
Galera cluster 설치 삽질과정 (완료)  (0) 2018.11.02
xtrabackup 설치해보기 1 - 의존성 리스트  (0) 2018.10.25
posted by 떠돌이늑대 2018. 11. 2. 16:12

설치환경은 Centos 1804

 

설치 DB 는 mariadb 10.3.10 으로 진행했다.

 

총 3대의 가상머신에서 진행을 하는 중이다.

 

node 1, 2, 3 이고

 

IP는 192.168.1.140~142로 할당을 해주었고,

 

Galera cluster 가 사용하는 포트는 4444, 4567,4568 이기 때문에

 

OS 방화벽 허용리스트에 포함해주었다.

 

관련 문서링크

 

 http://galeracluster.com/documentation-webpages/firewallsettings.html

 

 

방화벽 포트 확인

 [root@galera2 ~]# firewall-cmd --list-ports
3306/tcp 4444/tcp 4567/tcp 4568/tcp

 

 

MariaDB 10.3.x 부터는 SST에서 xtrabackup-v2를 지원하지 않기 때문에

mariabackup을 사용하기로 정했다.

 

 

node 1의

my.cnf 에

 

아래와 같이 설정을 추가해주었다.

[galera]
# Mandatory settings
wsrep_on=ON
wsrep_provider=/maria/app/lib/galera/libgalera_smm.so
wsrep_provider_options=gcache.size=128M
wsrep_cluster_address="gcomm://"
wsrep_cluster_name=cfpc_galera
wsrep_node_address=192.168.1.140
wsrep_node_name=galera1
wsrep_sst_receive_address=192.168.1.140:4444
wsrep_sst_method=mariabackup
wsrep_sst_auth=galera:123qwe!@#QWE
binlog_format=ROW
innodb_autoinc_lock_mode=2
default_storage_engine=InnoDB

 

그리고 Mariadb 데몬을 실행시켜주었다.

 

/etc/init.d/maria start --wsrep-new-cluster

 

그리고 DB에 접속해서

상태를 보았다.

MariaDB [(none)]> show status like 'wsrep%';
+------------------------------+----------------------------------------+
| Variable_name                | Value                                  |
+------------------------------+----------------------------------------+
| wsrep_apply_oooe           | 0.000000                               |
| wsrep_apply_oool            | 0.000000                               |
| wsrep_apply_window        | 0.000000                               |
| wsrep_causal_reads          | 0                                         |
| wsrep_cert_deps_distance  | 0.000000                               |
| wsrep_cert_index_size       | 0                                         |
| wsrep_cert_interval          | 0.000000                               |
| wsrep_cluster_conf_id       | 1                                         |
| wsrep_cluster_size           | 1                                         |
| wsrep_cluster_state_uuid   | 07112bf5-dce0-11e8-bd0f-fe56e4bea91e   |
| wsrep_cluster_status        | Primary                                |
| wsrep_cluster_weight       | 1                                        |
| wsrep_commit_oooe        | 0.000000                              |
| wsrep_commit_oool         | 0.000000                              |
| wsrep_commit_window      | 0.000000                             |
| wsrep_connected             | ON                                     |
| wsrep_desync_count         | 0                                        |
| wsrep_evs_delayed           |                                          |
| wsrep_evs_evict_list          |                                        |
| wsrep_evs_repl_latency     | 2.6e-06/5.08e-06/8.2e-06/2.22207e-06/5 |
| wsrep_evs_state              | OPERATIONAL                            |
| wsrep_flow_control_paused    | 0.000000                               |
| wsrep_flow_control_paused_ns | 0                                      |
| wsrep_flow_control_recv      | 0                                      |
| wsrep_flow_control_sent      | 0                                      |
| wsrep_gcomm_uuid             | 9dcdbc95-de36-11e8-92d7-9a829c5cb06a   |
| wsrep_incoming_addresses     | 192.168.1.140:3306                     |
| wsrep_last_committed         | 4                                      |
| wsrep_local_bf_aborts         | 0                                      |
| wsrep_local_cached_downto    | 18446744073709551615                   |
| wsrep_local_cert_failures    | 0                                      |
| wsrep_local_commits          | 0                                      |
| wsrep_local_index            | 0                                      |
| wsrep_local_recv_queue       | 0                                      |
| wsrep_local_recv_queue_avg   | 0.500000                               |
| wsrep_local_recv_queue_max   | 2                                      |
| wsrep_local_recv_queue_min   | 0                                      |
| wsrep_local_replays          | 0                                      |
| wsrep_local_send_queue       | 0                                      |
| wsrep_local_send_queue_avg   | 0.000000                               |
| wsrep_local_send_queue_max   | 1                                      |
| wsrep_local_send_queue_min   | 0                                      |
| wsrep_local_state            | 4                                      |
| wsrep_local_state_comment    | Synced                                 |
| wsrep_local_state_uuid       | 07112bf5-dce0-11e8-bd0f-fe56e4bea91e   |
| wsrep_open_connections       | 0                                      |
| wsrep_open_transactions      | 0                                      |
| wsrep_protocol_version       | 9                                      |
| wsrep_provider_name          | Galera                                 |
| wsrep_provider_vendor        | Codership Oy <info@codership.com>      |
| wsrep_provider_version       | 25.3.24(r3825)                         |
| wsrep_ready                  | ON                                     |
| wsrep_received               | 2                                      |
| wsrep_received_bytes         | 144                                    |
| wsrep_repl_data_bytes        | 0                                      |
| wsrep_repl_keys              | 0                                      |
| wsrep_repl_keys_bytes        | 0                                      |
| wsrep_repl_other_bytes       | 0                                      |
| wsrep_replicated             | 0                                      |
| wsrep_replicated_bytes       | 0                                      |
| wsrep_thread_count           | 2                                      |
+------------------------------+----------------------------------------+
61 rows in set (0.003 sec)

 

node1 에서는 잘 실행되는 것 같다 .

 

node2에서

 

my.cnf에 아래와 같이 추가를 해주었다.

 

[galera]
# Mandatory settings
wsrep_on=ON
wsrep_provider=/maria/app/lib/galera/libgalera_smm.so
wsrep_provider_options=gcache.size=128M
wsrep_cluster_address="gcomm://192.168.1.140:4567"
wsrep_cluster_name=cfpc_galera
wsrep_sst_receive_address=192.168.1.141:4444
wsrep_node_address=192.168.1.141
wsrep_node_name=galera2
wsrep_sst_method=mariabackup
wsrep_sst_auth=galera:123qwe!@#QWE
binlog_format=ROW
innodb_autoinc_lock_mode=2
default_storage_engine=InnoDB

 

DB 구동이 실패했다.

 

 [root@galera2 ~]# systemctl start maria
Job for maria.service failed because the control process exited with error code. See "systemctl status maria.service" and "journalctl -xe" for details.

 

journalctl 을 이용해보았다.

 

-- Unit maria.service has begun starting up.
Nov 02 10:13:05 galera2 maria[2521]: Starting MariaDB.181102 10:13:05 mysqld_safe Logging to '/maria_log/error/maria.err'.
Nov 02 10:13:05 galera2 maria[2521]: 181102 10:13:05 mysqld_safe Starting mysqld daemon with databases from /dbdata
Nov 02 10:13:27 galera2 maria[2521]: .................... ERROR!
Nov 02 10:13:27 galera2 systemd[1]: maria.service: control process exited, code=exited status=1
Nov 02 10:13:27 galera2 systemd[1]: Failed to start LSB: start and stop MariaDB.
-- Subject: Unit maria.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit maria.service has failed.
--
-- The result is failed.

 

maria.err 에 기록이 되었단다. 열어보자.

 

2018-11-02 10:13:08 0 [Note] WSREP: Read nil XID from storage engines, skipping position init
2018-11-02 10:13:08 0 [Note] WSREP: wsrep_load(): loading provider library '/maria/app/lib/galera/libgalera_smm.so'
2018-11-02 10:13:08 0 [Note] WSREP: wsrep_load(): Galera 25.3.24(r3825) by Codership Oy <info@codership.com> loaded successfully.
2018-11-02 10:13:08 0 [Note] WSREP: CRC-32C: using hardware acceleration.
2018-11-02 10:13:08 0 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 1
2018-11-02 10:13:08 0 [Note] WSREP: Passing config to GCS: base_dir = /dbdata/; base_host = 192.168.1.141; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /dbdata/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /dbdata//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false
2018-11-02 10:13:08 0 [Note] WSREP: GCache history reset: 07112bf5-dce0-11e8-bd0f-fe56e4bea91e:0 -> 00000000-0000-0000-0000-000000000000:-1
2018-11-02 10:13:08 0 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
2018-11-02 10:13:08 0 [Note] WSREP: wsrep_sst_grab()
2018-11-02 10:13:08 0 [Note] WSREP: Start replication
2018-11-02 10:13:08 0 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
2018-11-02 10:13:08 0 [Note] WSREP: protonet asio version 0
2018-11-02 10:13:08 0 [Note] WSREP: Using CRC-32C for message checksums.
2018-11-02 10:13:08 0 [Note] WSREP: backend: asio
2018-11-02 10:13:08 0 [Note] WSREP: gcomm thread scheduling priority set to other:0
2018-11-02 10:13:08 0 [Warning] WSREP: access file(/dbdata//gvwstate.dat) failed(No such file or directory)
2018-11-02 10:13:08 0 [Note] WSREP: restore pc from disk failed
2018-11-02 10:13:08 0 [Note] WSREP: GMCast version 0
2018-11-02 10:13:08 0 [Note] WSREP: (75a27987, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
2018-11-02 10:13:08 0 [Note] WSREP: (75a27987, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
2018-11-02 10:13:08 0 [Note] WSREP: EVS version 0
2018-11-02 10:13:08 0 [Note] WSREP: gcomm: connecting to group 'cfpc_galera', peer '192.168.1.140:4567'
2018-11-02 10:13:08 0 [Note] WSREP: (75a27987, 'tcp://0.0.0.0:4567') connection established to 9dcdbc95 tcp://192.168.1.140:4567
2018-11-02 10:13:08 0 [Note] WSREP: (75a27987, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
2018-11-02 10:13:09 0 [Note] WSREP: declaring 9dcdbc95 at tcp://192.168.1.140:4567 stable
2018-11-02 10:13:09 0 [Note] WSREP: Node 9dcdbc95 state prim

2018-11-02 10:13:09 0 [Note] WSREP: view(view_id(PRIM,75a27987,4) memb {
        75a27987,0
        9dcdbc95,0
} joined {
} left {
} partitioned {
})
2018-11-02 10:13:09 0 [Note] WSREP: save pc into disk
2018-11-02 10:13:09 0 [Note] WSREP: gcomm: connected
2018-11-02 10:13:09 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
2018-11-02 10:13:09 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
2018-11-02 10:13:09 0 [Note] WSREP: Opened channel 'cfpc_galera'
2018-11-02 10:13:09 0 [Note] WSREP: Waiting for SST to complete.
2018-11-02 10:13:09 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
2018-11-02 10:13:09 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 763ba9e2-de3c-11e8-9ed7-8a60fe20a71d
2018-11-02 10:13:09 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 763ba9e2-de3c-11e8-9ed7-8a60fe20a71d
2018-11-02 10:13:09 0 [Note] WSREP: STATE EXCHANGE: got state msg: 763ba9e2-de3c-11e8-9ed7-8a60fe20a71d from 0 (galera2)
2018-11-02 10:13:09 0 [Note] WSREP: STATE EXCHANGE: got state msg: 763ba9e2-de3c-11e8-9ed7-8a60fe20a71d from 1 (galera1)
2018-11-02 10:13:09 0 [Note] WSREP: Quorum results:
        version    = 4,
        component  = PRIMARY,
        conf_id    = 3,
        members    = 1/2 (joined/total),
        act_id     = 4,
        last_appl. = -1,
        protocols  = 0/9/3 (gcs/repl/appl),
        group UUID = 07112bf5-dce0-11e8-bd0f-fe56e4bea91e
2018-11-02 10:13:09 0 [Note] WSREP: Flow-control interval: [23, 23]
2018-11-02 10:13:09 0 [Note] WSREP: Trying to continue unpaused monitor
2018-11-02 10:13:09 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 4)
2018-11-02 10:13:09 2 [Note] WSREP: State transfer required:
        Group state: 07112bf5-dce0-11e8-bd0f-fe56e4bea91e:4
        Local state: 00000000-0000-0000-0000-000000000000:-1
2018-11-02 10:13:09 2 [Note] WSREP: New cluster view: global state: 07112bf5-dce0-11e8-bd0f-fe56e4bea91e:4, view# 4: Primary, number of nodes: 2, my index: 0, protocol version 3
2018-11-02 10:13:09 2 [Warning] WSREP: Gap in state sequence. Need state transfer.
2018-11-02 10:13:09 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'joiner' --address '192.168.1.141:4444' --datadir '/dbdata/'   --parent '2998' --binlog '/maria_log/binary/maria-bin'  '''
WSREP_SST: [INFO] Streaming with xbstream (20181102 10:13:10.481)
WSREP_SST: [INFO] Using socat as streamer (20181102 10:13:10.486)
WSREP_SST: [INFO] Stale sst_in_progress file: /dbdata//sst_in_progress (20181102 10:13:10.494)
WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | mbstream -x; RC=( ${PIPESTATUS[@]} ) (20181102 10:13:10.564)
2018-11-02 10:13:10 2 [Note] WSREP: Prepared SST request: mariabackup|192.168.1.141:4444/xtrabackup_sst//1
2018-11-02 10:13:10 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2018-11-02 10:13:10 2 [Note] WSREP: REPL Protocols: 9 (4, 2)
2018-11-02 10:13:10 2 [Note] WSREP: Assign initial position for certification: 4, protocol version: 4
2018-11-02 10:13:10 0 [Note] WSREP: Service thread queue flushed.

2018-11-02 10:13:10 2 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (07112bf5-dce0-11e8-bd0f-fe56e4bea91e): 1 (Operation not permitted)
         at galera/src/replicator_str.cpp:prepare_for_IST():482. IST will be unavailable.
2018-11-02 10:13:10 0 [Note] WSREP: Member 0.0 (galera2) requested state transfer from '*any*'. Selected 1.0 (galera1)(SYNCED) as donor.
2018-11-02 10:13:10 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 4)
2018-11-02 10:13:10 2 [Note] WSREP: Requesting state transfer: success, donor: 1
2018-11-02 10:13:10 2 [Note] WSREP: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 07112bf5-dce0-11e8-bd0f-fe56e4bea91e:4
WSREP_SST: [INFO] WARNING: Stale temporary SST directory: /dbdata//.sst from previous state transfer. Removing (20181102 10:13:11.434)
WSREP_SST: [INFO] Proceeding with SST (20181102 10:13:11.441)
WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | mbstream -x; RC=( ${PIPESTATUS[@]} ) (20181102 10:13:11.444)
WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20181102 10:13:11.452)
removed ‘/dbdata/aria_log_control’
removed ‘/dbdata/aria_log.00000001’
removed ‘/dbdata/ibdata1’
removed ‘/dbdata/ibdata2’
removed ‘/dbdata/ibdata3’
removed ‘/dbdata/ibdata4’
removed ‘/dbdata/ibdata5’
removed ‘/dbdata/ib_logfile1’
removed ‘/dbdata/ib_logfile2’
removed ‘/dbdata/ib_logfile3’
removed ‘/dbdata/ib_logfile4’
WSREP_SST: [INFO] Cleaning the binlog directory /maria_log/binary as well (20181102 10:13:11.473)
removed ‘/maria_log/binary/maria-bin.000001’
WSREP_SST: [INFO] Waiting for SST streaming to complete! (20181102 10:13:11.480)
2018-11-02 10:13:11 0 [Note] WSREP: (75a27987, 'tcp://0.0.0.0:4567') turning message relay requesting off
2018-11-02 10:13:21 0 [Warning] WSREP: 1.0 (galera1): State transfer to 0.0 (galera2) failed: -22 (Invalid argument)
2018-11-02 10:13:21 0 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():737: Will never receive state. Need to abort.
2018-11-02 10:13:21 0 [Note] WSREP: gcomm: terminating thread
2018-11-02 10:13:21 0 [Note] WSREP: gcomm: joining thread
2018-11-02 10:13:21 0 [Note] WSREP: gcomm: closing backend
WSREP_SST: [ERROR] xtrabackup_checkpoints missing, failed innobackupex/SST on donor (20181102 10:13:21.557)
WSREP_SST: [ERROR] Cleanup after exit with status:2 (20181102 10:13:21.609)
2018-11-02 10:13:21 0 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'joiner' --address '192.168.1.141:4444' --datadir '/dbdata/'   --parent '2998' --binlog '/maria_log/binary/maria-bin'  '': 2 (No such file or directory)
2018-11-02 10:13:21 0 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script.
2018-11-02 10:13:21 0 [ERROR] WSREP: SST failed: 2 (No such file or directory)
2018-11-02 10:13:21 0 [ERROR] Aborting

 

이러한 것들이 기록이 되었다.

 

 

그리고 node1에서 이러한 기록이 보인다.

 

innobackup.backup.log 파일을 열어본다.

 

181102 14:51:08 Connecting to MySQL server host: localhost, user: galera, password: set, port: not set, socket: /tmp/maria.sock
Failed to connect to MySQL server: Access denied for user 'galera'@'localhost' (using password: YES).

 

계정 문제?

 

이미 계정을 생성해주었고, 모든권한을 주었는데... 엑세스 거부라니...뭘까...

 

node1에서

 

mysql 접속한 다음.

 

drop user galera@localhost;

drop user  galera@127.0.0.1;

drop user galera@'%';

 

를 한다음

 

다시 생성해보았다.

 

그리고 node2에서 

systemctl start maria

된다....

 

 

node3에서 같은 문제가 발생했지만,

 

node1을 systemctl stop maria를 하여

 

중지한 다음 my.cnf 에

wsrep_cluster_address="gcomm://" 에 모든 노드의 IP를 추가해준다음

 

node1에서

 

systemctl start maria

 

서비스가 시작되고 갈레라클러스터가 제대로 작동된것을 확인후

 

node2에서 서비스 중지 후 my.cnf의

wsrep_cluster_address="gcomm://" 부분을 node1과 node3으로 변경해서

 

서비스를 시작 정상 작동 후 갈레라클러스터도 확인 정상

 

node3을 시작했더니.... 되더라..

 

MariaDB [(none)]> show status like 'wsrep_%';
+------------------------------+----------------------------------------------------------+
| Variable_name                | Value                                                    |
+------------------------------+----------------------------------------------------------+
| wsrep_apply_oooe             | 0.000000                                                 |
| wsrep_apply_oool             | 0.000000                                                 |
| wsrep_apply_window           | 0.000000                                                 |
| wsrep_causal_reads           | 0                                                        |
| wsrep_cert_deps_distance     | 0.000000                                                 |
| wsrep_cert_index_size        | 0                                                        |
| wsrep_cert_interval          | 0.000000                                                 |
| wsrep_cluster_conf_id        | 27                                                       |
| wsrep_cluster_size           | 3                                                        |
| wsrep_cluster_state_uuid     | ed7ec725-de61-11e8-9a68-26599eda2ee7                     |
| wsrep_cluster_status         | Primary                                                  |
| wsrep_cluster_weight         | 3                                                        |
| wsrep_commit_oooe            | 0.000000                                                 |
| wsrep_commit_oool            | 0.000000                                                 |
| wsrep_commit_window          | 0.000000                                                 |
| wsrep_connected              | ON                                                       |
| wsrep_desync_count           | 0                                                        |
| wsrep_evs_delayed            |                                                          |
| wsrep_evs_evict_list         |                                                          |
| wsrep_evs_repl_latency       | 0/0/0/0/0                                                |
| wsrep_evs_state              | OPERATIONAL                                              |
| wsrep_flow_control_paused    | 0.000000                                                 |
| wsrep_flow_control_paused_ns | 0                                                        |
| wsrep_flow_control_recv      | 0                                                        |
| wsrep_flow_control_sent      | 0                                                        |
| wsrep_gcomm_uuid             | ae0b4de8-de7d-11e8-bb50-3773face4542                     |
| wsrep_incoming_addresses     | 192.168.1.140:3306,192.168.1.141:3306,192.168.1.142:3306 |
| wsrep_last_committed         | 29                                                       |
| wsrep_local_bf_aborts        | 0                                                        |
| wsrep_local_cached_downto    | 18446744073709551615                                     |
| wsrep_local_cert_failures    | 0                                                        |
| wsrep_local_commits          | 0                                                        |
| wsrep_local_index            | 2                                                        |
| wsrep_local_recv_queue       | 0                                                        |
| wsrep_local_recv_queue_avg   | 0.000000                                                 |
| wsrep_local_recv_queue_max   | 1                                                        |
| wsrep_local_recv_queue_min   | 0                                                        |
| wsrep_local_replays          | 0                                                        |
| wsrep_local_send_queue       | 0                                                        |
| wsrep_local_send_queue_avg   | 0.000000                                                 |
| wsrep_local_send_queue_max   | 1                                                        |
| wsrep_local_send_queue_min   | 0                                                        |
| wsrep_local_state            | 4                                                        |
| wsrep_local_state_comment    | Synced                                                   |
| wsrep_local_state_uuid       | ed7ec725-de61-11e8-9a68-26599eda2ee7                     |
| wsrep_open_connections       | 0                                                        |
| wsrep_open_transactions      | 0                                                        |
| wsrep_protocol_version       | 9                                                        |
| wsrep_provider_name          | Galera                                                   |
| wsrep_provider_vendor        | Codership Oy <info@codership.com>                        |
| wsrep_provider_version       | 25.3.24(r3825)                                           |
| wsrep_ready                  | ON                                                       |
| wsrep_received               | 3                                                        |
| wsrep_received_bytes         | 296                                                      |
| wsrep_repl_data_bytes        | 0                                                        |
| wsrep_repl_keys              | 0                                                        |
| wsrep_repl_keys_bytes        | 0                                                        |
| wsrep_repl_other_bytes       | 0                                                        |
| wsrep_replicated             | 0                                                        |
| wsrep_replicated_bytes       | 0                                                        |
| wsrep_thread_count           | 2                                                        |
+------------------------------+----------------------------------------------------------+
61 rows in set (0.003 sec) 

 

 

 

험난하다 험난해...

 

가상머신을 다시 만들어서 해봐야겠다.

 

 

된다!

 

'컴이야기 > MariaDB' 카테고리의 다른 글

Maxscale 2.3.6  (1) 2019.07.12
Galera Cluster error  (0) 2019.06.14
Galera Cluster Replication.  (0) 2019.03.18
maxscale 을 사용해보자. [진행중]  (0) 2018.11.07
xtrabackup 설치해보기 1 - 의존성 리스트  (0) 2018.10.25
posted by 떠돌이늑대 2018. 10. 25. 09:35

Galera cluster를 이용해보기 위해  replication 도구로 xtrabackup을

 

사용해보기로 했다.

 

OS 는 Centos 7.5 1804 버전이고 minimal 로 설치한다음

 

mariadb는 10.3.9로 설치를 했다.

 

xtrabackup 은 mariadb에 기본 포함되어 있지 않기 때문에,

 

별도로 rpm 패키지로 설치를 진행했다.

 

의존성 문제가 나왔고, 하나하나 구해보았으나,

 

의존성의 의존성이 걸리는 심각한 정신적인 타격이 와서

 

yum을 이용하기로 하고, 의존성이 걸린 패키지들을

 

전부 다운로드 하는 방식으로 진행했다.

 

우선, xtrabackup 을 설치하기 위해서는 아래와 같은

 

rpm이 필요하다.

 

 

 

 

패키지들을 지정된 디렉토리로 다운로드 받고,

 

전부 설치를 진행해보니, 의존성 문제없이 설치가 되었다.

 

 

'컴이야기 > MariaDB' 카테고리의 다른 글

Maxscale 2.3.6  (1) 2019.07.12
Galera Cluster error  (0) 2019.06.14
Galera Cluster Replication.  (0) 2019.03.18
maxscale 을 사용해보자. [진행중]  (0) 2018.11.07
Galera cluster 설치 삽질과정 (완료)  (0) 2018.11.02