We use the same configuration files for IoTDB DataNode and Standalone version, all under the conf.
datanode-env.sh/bat：Environment configurations, in which we could set the memory allocation of DataNode and Standalone.
iotdb-datanode.properties：IoTDB system configurations.
For the convenience of users, IoTDB provides users with hot modification function, that is, modifying some configuration parameters in iotdb-datanode.properties during the system operation and applying them to the system immediately.
In the parameters described below, these parameters whose way of Effective is hot-load support hot modification.
Trigger way: The client sends the command(sql) load configuration or set configuration to the IoTDB server.
The environment configuration file is mainly used to configure the Java environment related parameters when DataNode is running, such as JVM related configuration. This part of the configuration is passed to the JVM when the DataNode starts.
The details of each parameter are as follows:
| Name | MEMORY_SIZE |
|---|
| Description | The minimum heap memory size that IoTDB DataNode will use when startup |
| Type | String |
| Default | The default is a half of the memory. |
| Effective | After restarting system |
| Name | ON_HEAP_MEMORY |
|---|
| Description | The heap memory size that IoTDB DataNode can use, Former Name: MAX_HEAP_SIZE |
| Type | String |
| Default | Calculate based on MEMORY_SIZE. |
| Effective | After restarting system |
| Name | OFF_HEAP_MEMORY |
|---|
| Description | The direct memory that IoTDB DataNode can use, Former Name: MAX_DIRECT_MEMORY_SIZE |
| Type | String |
| Default | Calculate based on MEMORY_SIZE. |
| Effective | After restarting system |
| Name | JMX_LOCAL |
|---|
| Description | JMX monitoring mode, configured as yes to allow only local monitoring, no to allow remote monitoring |
| Type | Enum String: "true", "false" |
| Default | true |
| Effective | After restarting system |
| Name | JMX_PORT |
|---|
| Description | JMX listening port. Please confirm that the port is not a system reserved port and is not occupied |
| Type | Short Int: [0,65535] |
| Default | 31999 |
| Effective | After restarting system |
| Name | JMX_IP |
|---|
| Description | JMX listening address. Only take effect if JMX_LOCAL=false. 0.0.0.0 is never allowed |
| Type | String |
| Default | 127.0.0.1 |
| Effective | After restarting system |
We STRONGLY RECOMMENDED you CHANGE the PASSWORD for the JMX remote connection.
The user and passwords are in ${IOTDB_CONF}/conf/jmx.password.
The permission definitions are in ${IOTDB_CONF}/conf/jmx.access.
| Name | dn_rpc_address |
|---|
| Description | The client rpc service listens on the address. |
| Type | String |
| Default | 0.0.0.0 |
| Effective | After restarting system |
| Name | dn_rpc_port |
|---|
| Description | The client rpc service listens on the port. |
| Type | Short Int : [0,65535] |
| Default | 6667 |
| Effective | After restarting system |
| Name | dn_internal_address |
|---|
| Description | DataNode internal service host/IP |
| Type | string |
| Default | 127.0.0.1 |
| Effective | Only allowed to be modified in first start up |
| Name | dn_internal_port |
|---|
| Description | DataNode internal service port |
| Type | int |
| Default | 10730 |
| Effective | Only allowed to be modified in first start up |
- dn_mpp_data_exchange_port
| Name | mpp_data_exchange_port |
|---|
| Description | MPP data exchange port |
| Type | int |
| Default | 10740 |
| Effective | Only allowed to be modified in first start up |
- dn_schema_region_consensus_port
| Name | dn_schema_region_consensus_port |
|---|
| Description | DataNode Schema replica communication port for consensus |
| Type | int |
| Default | 10750 |
| Effective | Only allowed to be modified in first start up |
- dn_data_region_consensus_port
| Name | dn_data_region_consensus_port |
|---|
| Description | DataNode Data replica communication port for consensus |
| Type | int |
| Default | 10760 |
| Effective | Only allowed to be modified in first start up |
- dn_join_cluster_retry_interval_ms
| Name | dn_join_cluster_retry_interval_ms |
|---|
| Description | The time of data node waiting for the next retry to join into the cluster |
| Type | long |
| Default | 5000 |
| Effective | After restarting system |
| Name | enable_thrift_ssl |
|---|
| Description | When enable_thrift_ssl is configured as true, SSL encryption will be used for communication through dn_rpc_port |
| Type | Boolean |
| Default | false |
| Effective | After restarting system |
| Name | enable_https |
|---|
| Description | REST Service Specifies whether to enable SSL configuration |
| Type | Boolean |
| Default | false |
| Effective | After restarting system |
| Name | key_store_path |
|---|
| Description | SSL certificate path |
| Type | String |
| Default | "" |
| Effective | After restarting system |
| Name | key_store_pwd |
|---|
| Description | SSL certificate password |
| Type | String |
| Default | "" |
| Effective | After restarting system |
| Name | dn_seed_config_node |
|---|
| Description | ConfigNode Address for DataNode to join cluster |
| Type | String |
| Default | 127.0.0.1:10710 |
| Effective | Only allowed to be modified in first start up |
- dn_rpc_thrift_compression_enable
| Name | dn_rpc_thrift_compression_enable |
|---|
| Description | Whether enable thrift's compression (using GZIP). |
| Type | Boolean |
| Default | false |
| Effective | After restarting system |
- dn_rpc_advanced_compression_enable
| Name | dn_rpc_advanced_compression_enable |
|---|
| Description | Whether enable thrift's advanced compression. |
| Type | Boolean |
| Default | false |
| Effective | After restarting system |
- dn_rpc_selector_thread_count
| Name | dn_rpc_selector_thread_count |
|---|
| Description | The number of rpc selector thread. |
| Type | int |
| Default | false |
| Effective | After restarting system |
- dn_rpc_min_concurrent_client_num
| Name | dn_rpc_min_concurrent_client_num |
|---|
| Description | Minimum concurrent rpc connections |
| Type | Short Int : [0,65535] |
| Description | 1 |
| Effective | After restarting system |
- dn_rpc_max_concurrent_client_num
| Name | dn_rpc_max_concurrent_client_num |
|---|
| Description | Max concurrent rpc connections |
| Type | Short Int : [0,65535] |
| Description | 65535 |
| Effective | After restarting system |
| Name | dn_thrift_max_frame_size |
|---|
| Description | Max size of bytes of each thrift RPC request/response |
| Type | Long |
| Unit | Byte |
| Default | 536870912 |
| Effective | After restarting system |
- dn_thrift_init_buffer_size
| Name | dn_thrift_init_buffer_size |
|---|
| Description | Initial size of bytes of buffer that thrift used |
| Type | long |
| Default | 1024 |
| Effective | After restarting system |
| Name | dn_connection_timeout_ms |
|---|
| Description | Thrift socket and connection timeout between nodes |
| Type | int |
| Default | 60000 |
| Effective | After restarting system |
- dn_core_client_count_for_each_node_in_client_manager
| Name | dn_core_client_count_for_each_node_in_client_manager |
|---|
| Description | Number of core clients routed to each node in a ClientManager |
| Type | int |
| Default | 200 |
| Effective | After restarting system |
- dn_max_client_count_for_each_node_in_client_manager
| Name | dn_max_client_count_for_each_node_in_client_manager |
|---|
| Description | Number of max clients routed to each node in a ClientManager |
| Type | int |
| Default | 300 |
| Effective | After restarting system |
| Name | dn_system_dir |
|---|
| Description | The directories of system files. It is recommended to use an absolute path. |
| Type | String |
| Default | data/datanode/system (Windows: data\datanode\system) |
| Effective | After restarting system |
| Name | dn_data_dirs |
|---|
| Description | The directories of data files. Multiple directories are separated by comma. The starting directory of the relative path is related to the operating system. It is recommended to use an absolute path. If the path does not exist, the system will automatically create it. |
| Type | String[] |
| Default | data/datanode/data (Windows: data\datanode\data) |
| Effective | After restarting system |
| Name | dn_multi_dir_strategy |
|---|
| Description | IoTDB's strategy for selecting directories for TsFile in tsfile_dir. You can use a simple class name or a full name of the class. The system provides the following three strategies:
1. SequenceStrategy: IoTDB selects the directory from tsfile_dir in order, traverses all the directories in tsfile_dir in turn, and keeps counting;
2. MaxDiskUsableSpaceFirstStrategy: IoTDB first selects the directory with the largest free disk space in tsfile_dir;
You can complete a user-defined policy in the following ways:
1. Inherit the org.apache.iotdb.db.storageengine.rescon.disk.strategy.DirectoryStrategy class and implement its own Strategy method;
2. Fill in the configuration class with the full class name of the implemented class (package name plus class name, UserDfineStrategyPackage);
3. Add the jar file to the project. |
| Type | String |
| Default | SequenceStrategy |
| Effective | hot-load |
| Name | dn_consensus_dir |
|---|
| Description | The directories of consensus files. It is recommended to use an absolute path. |
| Type | String |
| Default | data/datanode/consensus |
| Effective | After restarting system |
| Name | dn_wal_dirs |
|---|
| Description | Write Ahead Log storage path. It is recommended to use an absolute path. |
| Type | String |
| Default | data/datanode/wal |
| Effective | After restarting system |
| Name | dn_tracing_dir |
|---|
| Description | The tracing root directory path. It is recommended to use an absolute path. |
| Type | String |
| Default | datanode/tracing |
| Effective | After restarting system |
| Name | dn_sync_dir |
|---|
| Description | The directories of sync files. It is recommended to use an absolute path. |
| Type | String |
| Default | data/datanode/sync |
| Effective | After restarting system |
GC log is off by default.
For performance tuning, you may want to collect the GC info.
To enable GC log, just add a parameter "printgc" when you start the DataNode.
nohup sbin/start-datanode.sh printgc >/dev/null 2>&1 &
Or
sbin\start-datanode.bat printgc
GC log is stored at IOTDB_HOME/logs/gc.log.
There will be at most 10 gc.log.* files and each one can reach to 10MB.
| Name | enable_rest_service |
|---|
| Description | Whether to enable the Rest service |
| Type | Boolean |
| Default | false |
| Effective | After restarting system |
| Name | rest_service_port |
|---|
| Description | The Rest service listens to the port number |
| Type | int32 |
| Default | 18080 |
| Effective | After restarting system |
| Name | enable_swagger |
|---|
| Description | Whether to enable swagger to display rest interface information |
| Type | Boolean |
| Default | false |
| Effective | After restarting system |
- rest_query_default_row_size_limit
| Name | rest_query_default_row_size_limit |
|---|
| Description | The maximum number of rows in a result set that can be returned by a query |
| Type | int32 |
| Default | 10000 |
| Effective | After restarting system |
| Name | cache_expire |
|---|
| Description | Expiration time for caching customer login information |
| Type | int32 |
| Default | 28800 |
| Effective | After restarting system |
| Name | cache_max_num |
|---|
| Description | The maximum number of users stored in the cache |
| Type | int32 |
| Default | 100 |
| Effective | After restarting system |
| Name | cache_init_num |
|---|
| Description | Initial cache capacity |
| Type | int32 |
| Default | 10 |
| Effective | After restarting system |
| Name | trust_store_path |
|---|
| Description | keyStore Password (optional) |
| Type | String |
| Default | "" |
| Effective | After restarting system |
| Name | trust_store_pwd |
|---|
| Description | trustStore Password (Optional) |
| Type | String |
| Default | "" |
| Effective | After restarting system |
| Name | idle_timeout |
|---|
| Description | SSL timeout duration, expressed in seconds |
| Type | int32 |
| Default | 5000 |
| Effective | After restarting system |
- dn_default_space_move_thresholds
| Name | dn_default_space_move_thresholds |
|---|
| Description | Version 1.3.0/1: Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY |
| Type | double |
| Default | 0.15 |
| Effective | hot-load |
- dn_default_space_usage_thresholds
| Name | dn_default_space_usage_thresholds |
|---|
| Description | Version 1.3.2: Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY |
| Type | double |
| Default | 0.85 |
| Effective | hot-load |
| Name | remote_tsfile_cache_dirs |
|---|
| Description | Cache directory stored locally in the cloud |
| Type | string |
| Default | data/datanode/data/cache |
| Effective | After restarting system |
- remote_tsfile_cache_page_size_in_kb
| Name | remote_tsfile_cache_page_size_in_kb |
|---|
| Description | Block size of locally cached files stored in the cloud |
| Type | int |
| Default | 20480 |
| Effective | After restarting system |
- remote_tsfile_cache_max_disk_usage_in_mb
| Name | remote_tsfile_cache_max_disk_usage_in_mb |
|---|
| Description | Maximum Disk Occupancy Size for Cloud Storage Local Cache |
| Type | long |
| Default | 51200 |
| Effective | After restarting system |
| Name | object_storage_type |
|---|
| Description | Cloud Storage Type |
| Type | string |
| Default | AWS_S3 |
| Effective | After restarting system |
| Name | object_storage_bucket |
|---|
| Description | Name of cloud storage bucket |
| Type | string |
| Default | iotdb_data |
| Effective | After restarting system |
| Name | object_storage_endpoiont |
|---|
| Description | endpoint of cloud storage |
| Type | string |
| Default | None |
| Effective | After restarting system |
- object_storage_access_key
| Name | object_storage_access_key |
|---|
| Description | Authentication information stored in the cloud: key |
| Type | string |
| Default | None |
| Effective | After restarting system |
- object_storage_access_secret
| Name | object_storage_access_secret |
|---|
| Description | Authentication information stored in the cloud: secret |
| Type | string |
| Default | None |
| Effective | After restarting system |
