Skip to main content

DataNode Configuration Parameters

About 6 min

DataNode Configuration Parameters

We use the same configuration files for IoTDB DataNode and Standalone version, all under the conf.

  • datanode-env.sh/bat:Environment configurations, in which we could set the memory allocation of DataNode and Standalone.

  • iotdb-datanode.properties:IoTDB DataNode system configurations.

Hot Modification Configuration

For the convenience of users, IoTDB provides users with hot modification function, that is, modifying some configuration parameters in iotdb-datanode.properties and iotdb-common.properties during the system operation and applying them to the system immediately.
In the parameters described below, these parameters whose way of Effective is hot-load support hot modification.

Trigger way: The client sends the command(sql) load configuration to the IoTDB server.

Environment Configuration File(datanode-env.sh/bat)open in new window

The environment configuration file is mainly used to configure the Java environment related parameters when DataNode is running, such as JVM related configuration. This part of the configuration is passed to the JVM when the DataNode starts.

The details of each parameter are as follows:

  • MEMORY_SIZE
NameMEMORY_SIZE
DescriptionThe minimum heap memory size that IoTDB DataNode will use when startup
TypeString
DefaultThe default is a half of the memory.
EffectiveAfter restarting system
  • ON_HEAP_MEMORY
NameON_HEAP_MEMORY
DescriptionThe heap memory size that IoTDB DataNode can use, Former Name: MAX_HEAP_SIZE
TypeString
DefaultCalculate based on MEMORY_SIZE.
EffectiveAfter restarting system
  • OFF_HEAP_MEMORY
NameOFF_HEAP_MEMORY
DescriptionThe direct memory that IoTDB DataNode can use, Former Name: MAX_DIRECT_MEMORY_SIZE
TypeString
DefaultCalculate based on MEMORY_SIZE.
EffectiveAfter restarting system
  • JMX_LOCAL
NameJMX_LOCAL
DescriptionJMX monitoring mode, configured as yes to allow only local monitoring, no to allow remote monitoring
TypeEnum String: "true", "false"
Defaulttrue
EffectiveAfter restarting system
  • JMX_PORT
NameJMX_PORT
DescriptionJMX listening port. Please confirm that the port is not a system reserved port and is not occupied
TypeShort Int: [0,65535]
Default31999
EffectiveAfter restarting system
  • JMX_IP
NameJMX_IP
DescriptionJMX listening address. Only take effect if JMX_LOCAL=false. 0.0.0.0 is never allowed
TypeString
Default127.0.0.1
EffectiveAfter restarting system

JMX Authorization

We STRONGLY RECOMMENDED you CHANGE the PASSWORD for the JMX remote connection.

The user and passwords are in ${IOTDB_CONF}/conf/jmx.password.

The permission definitions are in ${IOTDB_CONF}/conf/jmx.access.

DataNode/Standalone Configuration File (iotdb-datanode.properties)

Data Node RPC Configuration

  • dn_rpc_address
Namedn_rpc_address
DescriptionThe client rpc service listens on the address.
TypeString
Default0.0.0.0
EffectiveAfter restarting system
  • dn_rpc_port
Namedn_rpc_port
DescriptionThe client rpc service listens on the port.
TypeShort Int : [0,65535]
Default6667
EffectiveAfter restarting system
  • dn_internal_address
Namedn_internal_address
DescriptionDataNode internal service host/IP
Typestring
Default127.0.0.1
EffectiveOnly allowed to be modified in first start up
  • dn_internal_port
Namedn_internal_port
DescriptionDataNode internal service port
Typeint
Default10730
EffectiveOnly allowed to be modified in first start up
  • dn_mpp_data_exchange_port
Namempp_data_exchange_port
DescriptionMPP data exchange port
Typeint
Default10740
EffectiveOnly allowed to be modified in first start up
  • dn_schema_region_consensus_port
Namedn_schema_region_consensus_port
DescriptionDataNode Schema replica communication port for consensus
Typeint
Default10750
EffectiveOnly allowed to be modified in first start up
  • dn_data_region_consensus_port
Namedn_data_region_consensus_port
DescriptionDataNode Data replica communication port for consensus
Typeint
Default10760
EffectiveOnly allowed to be modified in first start up
  • dn_join_cluster_retry_interval_ms
Namedn_join_cluster_retry_interval_ms
DescriptionThe time of data node waiting for the next retry to join into the cluster
Typelong
Default5000
EffectiveAfter restarting system

SSL Configuration

  • enable_thrift_ssl
Nameenable_thrift_ssl
DescriptionWhen enable_thrift_ssl is configured as true, SSL encryption will be used for communication through dn_rpc_port
TypeBoolean
Defaultfalse
EffectiveAfter restarting system
  • enable_https
Namecache_init_num
DescriptionREST Service Specifies whether to enable SSL configuration
TypeBoolean
Defaultfalse
EffectiveAfter restarting system
  • key_store_path
Namekey_store_path
DescriptionSSL certificate path
TypeString
Default""
EffectiveAfter restarting system
  • key_store_pwd
Namekey_store_pwd
DescriptionSSL certificate password
TypeString
Default""
EffectiveAfter restarting system

Target Config Nodes

  • dn_seed_config_node
Namedn_seed_config_node
DescriptionConfigNode Address for DataNode to join cluster
TypeString
Default127.0.0.1:10710
EffectiveOnly allowed to be modified in first start up

Connection Configuration

  • dn_rpc_thrift_compression_enable
Namedn_rpc_thrift_compression_enable
DescriptionWhether enable thrift's compression (using GZIP).
TypeBoolean
Defaultfalse
EffectiveAfter restarting system
  • dn_rpc_advanced_compression_enable
Namedn_rpc_advanced_compression_enable
DescriptionWhether enable thrift's advanced compression.
TypeBoolean
Defaultfalse
EffectiveAfter restarting system
  • dn_rpc_selector_thread_count
Namedn_rpc_selector_thread_count
DescriptionThe number of rpc selector thread.
Typeint
Defaultfalse
EffectiveAfter restarting system
  • dn_rpc_min_concurrent_client_num
Namedn_rpc_min_concurrent_client_num
DescriptionMinimum concurrent rpc connections
TypeShort Int : [0,65535]
Description1
EffectiveAfter restarting system
  • dn_rpc_max_concurrent_client_num
Namedn_rpc_max_concurrent_client_num
DescriptionMax concurrent rpc connections
TypeShort Int : [0,65535]
Description65535
EffectiveAfter restarting system
  • dn_thrift_max_frame_size
Namedn_thrift_max_frame_size
DescriptionMax size of bytes of each thrift RPC request/response
TypeLong
UnitByte
Default536870912
EffectiveAfter restarting system
  • dn_thrift_init_buffer_size
Namedn_thrift_init_buffer_size
DescriptionInitial size of bytes of buffer that thrift used
Typelong
Default1024
EffectiveAfter restarting system
  • dn_connection_timeout_ms
Namedn_connection_timeout_ms
DescriptionThrift socket and connection timeout between nodes
Typeint
Default60000
EffectiveAfter restarting system
  • dn_core_client_count_for_each_node_in_client_manager
Namedn_core_client_count_for_each_node_in_client_manager
DescriptionNumber of core clients routed to each node in a ClientManager
Typeint
Default200
EffectiveAfter restarting system
  • dn_max_client_count_for_each_node_in_client_manager
Namedn_max_client_count_for_each_node_in_client_manager
DescriptionNumber of max clients routed to each node in a ClientManager
Typeint
Default300
EffectiveAfter restarting system

Dictionary Configuration

  • dn_system_dir
Namedn_system_dir
DescriptionThe directories of system files. It is recommended to use an absolute path.
TypeString
Defaultdata/datanode/system (Windows: data\datanode\system)
EffectiveAfter restarting system
  • dn_data_dirs
Namedn_data_dirs
DescriptionThe directories of data files. Multiple directories are separated by comma. The starting directory of the relative path is related to the operating system. It is recommended to use an absolute path. If the path does not exist, the system will automatically create it.
TypeString[]
Defaultdata/datanode/data (Windows: data\datanode\data)
EffectiveAfter restarting system
  • dn_multi_dir_strategy
Namedn_multi_dir_strategy
DescriptionIoTDB's strategy for selecting directories for TsFile in tsfile_dir. You can use a simple class name or a full name of the class. The system provides the following three strategies:
1. SequenceStrategy: IoTDB selects the directory from tsfile_dir in order, traverses all the directories in tsfile_dir in turn, and keeps counting;
2. MaxDiskUsableSpaceFirstStrategy: IoTDB first selects the directory with the largest free disk space in tsfile_dir;
You can complete a user-defined policy in the following ways:
1. Inherit the org.apache.iotdb.db.storageengine.rescon.disk.strategy.DirectoryStrategy class and implement its own Strategy method;
2. Fill in the configuration class with the full class name of the implemented class (package name plus class name, UserDfineStrategyPackage);
3. Add the jar file to the project.
TypeString
DefaultSequenceStrategy
Effectivehot-load
  • dn_consensus_dir
Namedn_consensus_dir
DescriptionThe directories of consensus files. It is recommended to use an absolute path.
TypeString
Defaultdata/datanode/consensus
EffectiveAfter restarting system
  • dn_wal_dirs
Namedn_wal_dirs
DescriptionWrite Ahead Log storage path. It is recommended to use an absolute path.
TypeString
Defaultdata/datanode/wal
EffectiveAfter restarting system
  • dn_tracing_dir
Namedn_tracing_dir
DescriptionThe tracing root directory path. It is recommended to use an absolute path.
TypeString
Defaultdatanode/tracing
EffectiveAfter restarting system
  • dn_sync_dir
Namedn_sync_dir
DescriptionThe directories of sync files. It is recommended to use an absolute path.
TypeString
Defaultdata/datanode/sync
EffectiveAfter restarting system

Metric Configuration

Enable GC log

GC log is off by default.
For performance tuning, you may want to collect the GC info.

To enable GC log, just add a parameter "printgc" when you start the DataNode.

nohup sbin/start-datanode.sh printgc >/dev/null 2>&1 &

Or

sbin\start-datanode.bat printgc

GC log is stored at IOTDB_HOME/logs/gc.log.
There will be at most 10 gc.log.* files and each one can reach to 10MB.

REST Service Configuration

  • enable_rest_service
Nameenable_rest_service
DescriptionWhether to enable the Rest service
TypeBoolean
Defaultfalse
EffectiveAfter restarting system
  • rest_service_port
Namerest_service_port
DescriptionThe Rest service listens to the port number
Typeint32
Default18080
EffectiveAfter restarting system
  • enable_swagger
Nameenable_swagger
DescriptionWhether to enable swagger to display rest interface information
TypeBoolean
Defaultfalse
EffectiveAfter restarting system
  • rest_query_default_row_size_limit
Namerest_query_default_row_size_limit
DescriptionThe maximum number of rows in a result set that can be returned by a query
Typeint32
Default10000
EffectiveAfter restarting system
  • cache_expire
Namecache_expire
DescriptionExpiration time for caching customer login information
Typeint32
Default28800
EffectiveAfter restarting system
  • cache_max_num
Namecache_max_num
DescriptionThe maximum number of users stored in the cache
Typeint32
Default100
EffectiveAfter restarting system
  • cache_init_num
Namecache_init_num
DescriptionInitial cache capacity
Typeint32
Default10
EffectiveAfter restarting system
  • trust_store_path
Nametrust_store_path
DescriptionkeyStore Password (optional)
TypeString
Default""
EffectiveAfter restarting system
  • trust_store_pwd
Nametrust_store_pwd
DescriptiontrustStore Password (Optional)
TypeString
Default""
EffectiveAfter restarting system
  • idle_timeout
Nameidle_timeout
DescriptionSSL timeout duration, expressed in seconds
Typeint32
Default5000
EffectiveAfter restarting system

Copyright © 2024 The Apache Software Foundation.
Apache and the Apache feather logo are trademarks of The Apache Software Foundation

Have a question? Connect with us on QQ, WeChat, or Slack. Join the community now.

We use Google Analytics to collect anonymous, aggregated usage information.