DataNode Configuration Parameters

We use the same configuration files for IoTDB DataNode and Standalone version, all under the conf.

• datanode-env.sh/bat：Environment configurations, in which we could set the memory allocation of DataNode and Standalone.

• iotdb-datanode.properties：IoTDB DataNode system configurations.

Hot Modification Configuration

For the convenience of users, IoTDB provides users with hot modification function, that is, modifying some configuration parameters in iotdb-datanode.properties and iotdb-common.properties during the system operation and applying them to the system immediately.
In the parameters described below, these parameters whose way of Effective is hot-load support hot modification.

Trigger way: The client sends the command(sql) load configuration to the IoTDB server.

datanode-env.sh/bat）

The environment configuration file is mainly used to configure the Java environment related parameters when DataNode is running, such as JVM related configuration. This part of the configuration is passed to the JVM when the DataNode starts.

The details of each parameter are as follows:

• MAX_HEAP_SIZE

Name	MAX_HEAP_SIZE
Description	The maximum heap memory size that IoTDB can use
Type	String
Default	On Linux or MacOS, the default is one quarter of the memory. On Windows, the default value for 32-bit systems is 512M, and the default for 64-bit systems is 2G.
Effective	After restarting system

• HEAP_NEWSIZE

Name	HEAP_NEWSIZE
Description	The minimum heap memory size that IoTDB will use when startup
Type	String
Default	On Linux or MacOS, the default is min{cores * 100M, one quarter of MAX_HEAP_SIZE}. On Windows, the default value for 32-bit systems is 512M, and the default for 64-bit systems is 2G.
Effective	After restarting system

• MAX_DIRECT_MEMORY_SIZE

Name	MAX_DIRECT_MEMORY_SIZE
Description	The max direct memory that IoTDB could use
Type	String
Default	Equal to the MAX_HEAP_SIZE
Effective	After restarting system

• JMX_LOCAL

Name	JMX_LOCAL
Description	JMX monitoring mode, configured as yes to allow only local monitoring, no to allow remote monitoring
Type	Enum String: "true", "false"
Default	true
Effective	After restarting system

• JMX_PORT

Name	JMX_PORT
Description	JMX listening port. Please confirm that the port is not a system reserved port and is not occupied
Type	Short Int: [0,65535]
Default	31999
Effective	After restarting system

• JMX_IP

Name	JMX_IP
Description	JMX listening address. Only take effect if JMX_LOCAL=false. 0.0.0.0 is never allowed
Type	String
Default	127.0.0.1
Effective	After restarting system

JMX Authorization

We STRONGLY RECOMMENDED you CHANGE the PASSWORD for the JMX remote connection.

The user and passwords are in ${IOTDB_CONF}/conf/jmx.password.

The permission definitions are in ${IOTDB_CONF}/conf/jmx.access.

DataNode/Standalone Configuration File (iotdb-datanode.properties)

Data Node RPC Configuration

• dn_rpc_address

Name	dn_rpc_address
Description	The client rpc service listens on the address.
Type	String
Default	0.0.0.0
Effective	After restarting system

• dn_rpc_port

Name	dn_rpc_port
Description	The client rpc service listens on the port.
Type	Short Int : [0,65535]
Default	6667
Effective	After restarting system

• dn_internal_address

Name	dn_internal_address
Description	DataNode internal service host/IP
Type	string
Default	127.0.0.1
Effective	Only allowed to be modified in first start up

• dn_internal_port

Name	dn_internal_port
Description	DataNode internal service port
Type	int
Default	10730
Effective	Only allowed to be modified in first start up

• dn_mpp_data_exchange_port

Name	mpp_data_exchange_port
Description	MPP data exchange port
Type	int
Default	10740
Effective	Only allowed to be modified in first start up

• dn_schema_region_consensus_port

Name	dn_schema_region_consensus_port
Description	DataNode Schema replica communication port for consensus
Type	int
Default	10750
Effective	Only allowed to be modified in first start up

• dn_data_region_consensus_port

Name	dn_data_region_consensus_port
Description	DataNode Data replica communication port for consensus
Type	int
Default	10760
Effective	Only allowed to be modified in first start up

• dn_join_cluster_retry_interval_ms

Name	dn_join_cluster_retry_interval_ms
Description	The time of data node waiting for the next retry to join into the cluster
Type	long
Default	5000
Effective	After restarting system

Target Config Nodes

• dn_target_config_node_list

Name	dn_target_config_node_list
Description	ConfigNode Address for DataNode to join cluster
Type	String
Default	127.0.0.1:10710
Effective	Only allowed to be modified in first start up

Connection Configuration

• dn_rpc_thrift_compression_enable

Name	dn_rpc_thrift_compression_enable
Description	Whether enable thrift's compression (using GZIP).
Type	Boolean
Default	false
Effective	After restarting system

• dn_rpc_advanced_compression_enable

Name	dn_rpc_advanced_compression_enable
Description	Whether enable thrift's advanced compression.
Type	Boolean
Default	false
Effective	After restarting system

• dn_rpc_selector_thread_count

Name	dn_rpc_selector_thread_count
Description	The number of rpc selector thread.
Type	int
Default	false
Effective	After restarting system

• dn_rpc_min_concurrent_client_num

Name	dn_rpc_min_concurrent_client_num
Description	Minimum concurrent rpc connections
Type	Short Int : [0,65535]
Description	1
Effective	After restarting system

• dn_rpc_max_concurrent_client_num

Name	dn_rpc_max_concurrent_client_num
Description	Max concurrent rpc connections
Type	Short Int : [0,65535]
Description	65535
Effective	After restarting system

• dn_thrift_max_frame_size

Name	dn_thrift_max_frame_size
Description	Max size of bytes of each thrift RPC request/response
Type	Long
Unit	Byte
Default	536870912
Effective	After restarting system

• dn_thrift_init_buffer_size

Name	dn_thrift_init_buffer_size
Description	Initial size of bytes of buffer that thrift used
Type	long
Default	1024
Effective	After restarting system

• dn_core_client_count_for_each_node_in_client_manager

Name	dn_core_client_count_for_each_node_in_client_manager
Description	Number of core clients routed to each node in a ClientManager
Type	int
Default	200
Effective	After restarting system

• dn_max_client_count_for_each_node_in_client_manager

Name	dn_max_client_count_for_each_node_in_client_manager
Description	Number of max clients routed to each node in a ClientManager
Type	int
Default	300
Effective	After restarting system

Dictionary Configuration

• dn_system_dir

Name	dn_system_dir
Description	The directories of system files. It is recommended to use an absolute path.
Type	String
Default	data/datanode/system (Windows: data\datanode\system)
Effective	After restarting system

• dn_data_dirs

Name	dn_data_dirs
Description	The directories of data files. Multiple directories are separated by comma. The starting directory of the relative path is related to the operating system. It is recommended to use an absolute path. If the path does not exist, the system will automatically create it.
Type	String[]
Default	data/datanode/data (Windows: data\datanode\data)
Effective	After restarting system

• dn_multi_dir_strategy

Name	dn_multi_dir_strategy
Description	IoTDB's strategy for selecting directories for TsFile in tsfile_dir. You can use a simple class name or a full name of the class. The system provides the following three strategies: 1. SequenceStrategy: IoTDB selects the directory from tsfile_dir in order, traverses all the directories in tsfile_dir in turn, and keeps counting; 2. MaxDiskUsableSpaceFirstStrategy: IoTDB first selects the directory with the largest free disk space in tsfile_dir; You can complete a user-defined policy in the following ways: 1. Inherit the org.apache.iotdb.db.storageengine.rescon.disk.strategy.DirectoryStrategy class and implement its own Strategy method; 2. Fill in the configuration class with the full class name of the implemented class (package name plus class name, UserDfineStrategyPackage); 3. Add the jar file to the project.
Type	String
Default	SequenceStrategy
Effective	hot-load

• dn_consensus_dir

Name	dn_consensus_dir
Description	The directories of consensus files. It is recommended to use an absolute path.
Type	String
Default	data/datanode/consensus
Effective	After restarting system

• dn_wal_dirs

Name	dn_wal_dirs
Description	Write Ahead Log storage path. It is recommended to use an absolute path.
Type	String
Default	data/datanode/wal
Effective	After restarting system

• dn_tracing_dir

Name	dn_tracing_dir
Description	The tracing root directory path. It is recommended to use an absolute path.
Type	String
Default	datanode/tracing
Effective	After restarting system

• dn_sync_dir

Name	dn_sync_dir
Description	The directories of sync files. It is recommended to use an absolute path.
Type	String
Default	data/datanode/sync
Effective	After restarting system

Metric Configuration

Enable GC log

GC log is off by default.
For performance tuning, you may want to collect the GC info.

To enable GC log, just add a parameter "printgc" when you start the DataNode.

nohup sbin/start-datanode.sh printgc >/dev/null 2>&1 &

Or

sbin\start-datanode.bat printgc

GC log is stored at IOTDB_HOME/logs/gc.log.
There will be at most 10 gc.log.* files and each one can reach to 10MB.

Storage engine configuration

• dn_default_space_move_thresholds

Name	dn_default_space_move_thresholds
Description	Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY
Type	double
Default	0.15
Effective	hot-load

• remote_tsfile_cache_dirs

Name	remote_tsfile_cache_dirs
Description	Cache directory stored locally in the cloud
Type	string
Default	data/datanode/data/cache
Effective	After restarting system

• remote_tsfile_cache_page_size_in_kb

Name	remote_tsfile_cache_page_size_in_kb
Description	Block size of locally cached files stored in the cloud
Type	int
Default	20480
Effective	After restarting system

• remote_tsfile_cache_max_disk_usage_in_mb

Name	remote_tsfile_cache_max_disk_usage_in_mb
Description	Maximum Disk Occupancy Size for Cloud Storage Local Cache
Type	long
Default	51200
Effective	After restarting system

• object_storage_type

Name	object_storage_type
Description	Cloud Storage Type
Type	string
Default	AWS_S3
Effective	After restarting system

• object_storage_bucket

Name	object_storage_bucket
Description	Name of cloud storage bucket
Type	string
Default	iotdb_data
Effective	After restarting system

• object_storage_endpoiont

Name	object_storage_endpoiont
Description	endpoint of cloud storage
Type	string
Default	None
Effective	After restarting system

• object_storage_access_key

Name	object_storage_access_key
Description	Authentication information stored in the cloud: key
Type	string
Default	None
Effective	After restarting system

• object_storage_access_secret

Name	object_storage_access_secret
Description	Authentication information stored in the cloud: secret
Type	string
Default	None
Effective	After restarting system