# Appendix 1: Configuration Parameters

Before starting to use IoTDB, you need to config the configuration files first. For your convenience, we have already set the default config in the files.

In total, we provide users three kinds of configurations module:

  • environment configuration file (iotdb-env.bat, iotdb-env.sh). The default configuration file for the environment configuration item. Users can configure the relevant system configuration items of JAVA-JVM in the file.

  • system configuration file (iotdb-engine.properties).

    • iotdb-engine.properties: The default configuration file for the IoTDB engine layer configuration item. Users can configure the IoTDB engine related parameters in the file, such as JDBC service listening port (rpc_port), unsequence data storage directory (unsequence_data_dir), etc. What's more, Users can configure the information about the TsFile, such as the data size written to the disk per time(group_size_in_byte).
  • log configuration file (logback.xml)

The configuration files of the three configuration items are located in the IoTDB installation directory: $IOTDB_HOME/conf folder.

# Hot Modification Configuration

For the convenience of users, IoTDB server provides users with hot modification function, that is, modifying some configuration parameters in iotdb engine. Properties during the system operation and applying them to the system immediately. In the parameters described below, these parameters whose way of Effective is trigger support hot modification.

Trigger way: The client sends the command load configuration to the IoTDB server. See Chapter 4 for the usage of the client.

# IoTDB Environment Configuration File

The environment configuration file is mainly used to configure the Java environment related parameters when IoTDB Server is running, such as JVM related configuration. This part of the configuration is passed to the JVM when the IoTDB Server starts. Users can view the contents of the environment configuration file by viewing the iotdb-env.sh (or iotdb-env.bat) file.

The detail of each variables are as follows:

  • MAX_HEAP_SIZE
Name MAX_HEAP_SIZE
Description The maximum heap memory size that IoTDB can use at startup.
Type String
Default On Linux or MacOS, the default is one quarter of the memory. On Windows, the default value for 32-bit systems is 512M, and the default for 64-bit systems is 2G.
Effective After restart system
  • HEAP_NEWSIZE
Name HEAP_NEWSIZE
Description The minimum heap memory size that IoTDB can use at startup.
Type String
Default On Linux or MacOS, the default is min{cores * 100M, one quarter of MAX_HEAP_SIZE}. On Windows, the default value for 32-bit systems is 512M, and the default for 64-bit systems is 2G.
Effective After restart system
  • JMX_LOCAL
Name JMX_LOCAL
Description JMX monitoring mode, configured as yes to allow only local monitoring, no to allow remote monitoring
Type Enum String: "true", "false"
Default true
Effective After restart system
  • JMX_PORT
Name JMX_PORT
Description JMX listening port. Please confirm that the port is not a system reserved port and is not occupied
Type Short Int: [0,65535]
Default 31999
Effective After restart system
  • JMX_IP
Name JMX_IP
Description JMX listening address. Only take effect if JMX_LOCAL=false. 0.0.0.0 is never allowed
Type String
Default 127.0.0.1
Effective After restart system

# JMX Authorization

We STRONGLY RECOMMENDED you CHANGE the PASSWORD for the JMX remote connection.

The user and passwords are in ${IOTDB_CONF}/conf/jmx.password.

The permission definitions are in ${IOTDB_CONF}/conf/jmx.access.

# IoTDB System Configuration File

# File Layer

  • compressor
Name compressor
Description Data compression method
Type Enum String : “UNCOMPRESSED”, “SNAPPY”
Default UNCOMPRESSED
Effective Trigger
  • group_size_in_byte
Name group_size_in_byte
Description The data size written to the disk per time
Type Int32
Default 134217728
Effective Trigger
  • page_size_in_byte
Name page_size_in_byte
Description The maximum size of a single page written in memory when each column in memory is written (in bytes)
Type Int32
Default 65536
Effective Trigger
  • max_number_of_points_in_page
Name max_number_of_points_in_page
Description The maximum number of data points (timestamps - valued groups) contained in a page
Type Int32
Default 1048576
Effective Trigger
  • max_degree_of_index_node
Name max_degree_of_index_node
Description The maximum degree of the metadata index tree (that is, the max number of each node's children)
Type Int32
Default 256
Effective Only allowed to be modified in first start up
  • max_string_length
Name max_string_length
Description The maximum length of a single string (number of character)
Type Int32
Default 128
Effective Trigger
  • time_series_data_type
Name time_series_data_type
Description Timestamp data type
Type Enum String: "INT32", "INT64"
Default Int64
Effective Trigger
  • time_encoder
Name time_encoder
Description Encoding type of time column
Type Enum String: “TS_2DIFF”,“PLAIN”,“RLE”
Default TS_2DIFF
Effective Trigger
  • value_encoder
Name value_encoder
Description Encoding type of value column
Type Enum String: “TS_2DIFF”,“PLAIN”,“RLE”
Default PLAIN
Effective Trigger
  • float_precision
Name float_precision
Description The precision of the floating point number.(The number of digits after the decimal point)
Type Int32
Default The default is 2 digits. Note: The 32-bit floating point number has a decimal precision of 7 bits, and the 64-bit floating point number has a decimal precision of 15 bits. If the setting is out of the range, it will have no practical significance.
Effective Trigger
  • bloomFilterErrorRate
Name bloomFilterErrorRate
Description The false positive rate of bloom filter in each TsFile. Bloom filter checks whether a given time series is in the tsfile before loading metadata. This can improve the performance of loading metadata and skip the tsfile that doesn't contain specified time series. If you want to learn more about its mechanism, you can refer to: wiki page of bloom filter (opens new window).
Type float, (0, 1)
Default 0.05
Effective After restart system

# Engine Layer

  • rpc_address
Name rpc_address
Description The jdbc service listens on the address.
Type String
Default "0.0.0.0"
Effective After restart system
  • rpc_port
Name rpc_port
Description The jdbc service listens on the port. Please confirm that the port is not a system reserved port and is not occupied.
Type Short Int : [0,65535]
Default 6667
Effective After restart system
  • rpc_thrift_compression_enable
Name rpc_thrift_compression_enable
Description Whether enable thrift's compression (using GZIP).
Type Boolean
Default false
Effective After restart system
  • rpc_advanced_compression_enable
Name rpc_advanced_compression_enable
Description Whether enable thrift's advanced compression.
Type Boolean
Default false
Effective After restart system
  • time_zone
Name time_zone
Description The time zone in which the server is located, the default is Beijing time (+8)
Type Time Zone String
Default +08:00
Effective Trigger
  • base_dir
Name base_dir
Description The IoTDB system folder. It is recommended to use an absolute path.
Type String
Default data
Effective After restart system
  • data_dirs
Name data_dirs
Description The directories of data files. Multiple directories are separated by comma. The starting directory of the relative path is related to the operating system. It is recommended to use an absolute path. If the path does not exist, the system will automatically create it.
Type String[]
Default data/data
Effective Trigger
  • wal_dir
Name wal_dir
Description Write Ahead Log storage path. It is recommended to use an absolute path.
Type String
Default data/wal
Effective After restart system
  • enable_wal
Name enable_wal
Description Whether to enable the pre-write log. The default value is true(enabled), and false means closed.
Type Bool
Default true
Effective Trigger
  • enable_mem_control
Name enable_mem_control
Description enable memory control to avoid OOM
Type Bool
Default true
Effective After restart system
  • memtable_size_threshold
Name memtable_size_threshold
Description max memtable size
Type Long
Default 1073741824
Effective when enable_mem_control is false & After restart system
  • avg_series_point_number_threshold
Name avg_series_point_number_threshold
Description max average number of point of each series in memtable
Type Int32
Default 10000
Effective After restart system
  • tsfile_size_threshold
Name tsfile_size_threshold
Description max tsfile size
Type Long
Default 536870912
Effective After restart system
  • enable_partition
Name enable_partition
Description Whether enable time partition for data, if disabled, all data belongs to partition 0
Type Bool
Default false
Effective Only allowed to be modified in first start up
  • partition_interval
Name partition_interval
Description Time range for dividing storage group, time series data will be divided into groups by this time range
Type Int64
Default 604800
Effective Only allowed to be modified in first start up
  • concurrent_writing_time_partition
Name concurrent_writing_time_partition
Description This config decides how many time partitions in a storage group can be inserted concurrently
For example, your partitionInterval is 86400 and you want to insert data in 5 different days,
Type Int32
Default 1
Effective After restart system
  • multi_dir_strategy
Name multi_dir_strategy
Description IoTDB's strategy for selecting directories for TsFile in tsfile_dir. You can use a simple class name or a full name of the class. The system provides the following three strategies:
1. SequenceStrategy: IoTDB selects the directory from tsfile_dir in order, traverses all the directories in tsfile_dir in turn, and keeps counting;
2. MaxDiskUsableSpaceFirstStrategy: IoTDB first selects the directory with the largest free disk space in tsfile_dir;
3. MinFolderOccupiedSpaceFirstStrategy: IoTDB prefers the directory with the least space used in tsfile_dir;
4. UserDfineStrategyPackage (user-defined policy)
You can complete a user-defined policy in the following ways:
1. Inherit the cn.edu.tsinghua.iotdb.conf.directories.strategy.DirectoryStrategy class and implement its own Strategy method;
2. Fill in the configuration class with the full class name of the implemented class (package name plus class name, UserDfineStrategyPackage);
3. Add the jar file to the project.
Type String
Default MaxDiskUsableSpaceFirstStrategy
Effective Trigger
  • tsfile_size_threshold
Name tsfile_size_threshold
Description When a TsFile size on the disk exceeds this threshold, the TsFile is closed and open a new TsFile to accept data writes. The unit is byte and the default value is 2G.
Type Int64
Default 536870912
Effective After restart system
  • tag_attribute_total_size
Name tag_attribute_total_size
Description The maximum persistence size of tags and attributes of each time series.
Type Int32
Default 700
Effective Only allowed to be modified in first start up
  • enable_partial_insert
Name enable_partial_insert
Description Whether continue to write other measurements if some measurements are failed in one insertion.
Type Bool
Default true
Effective After restart system
  • mtree_snapshot_interval
Name mtree_snapshot_interval
Description The least interval line numbers of mlog.txt when creating a checkpoint and saving snapshot of MTree. Unit: line numbers
Type Int32
Default 100000
Effective After restart system
  • flush_wal_threshold
Name flush_wal_threshold
Description After the WAL reaches this value, it is flushed to disk, and it is possible to lose at most flush_wal_threshold operations.
Type Int32
Default 10000
Effective Trigger
  • force_wal_period_in_ms
Name force_wal_period_in_ms
Description The period during which the log is periodically forced to flush to disk(in milliseconds)
Type Int32
Default 10
Effective Trigger
  • fetch_size
Name fetch_size
Description The amount of data read each time in batch (the number of data strips, that is, the number of different timestamps.)
Type Int32
Default 10000
Effective After restart system
  • merge_concurrent_threads
Name merge_concurrent_threads
Description THe max threads which can be used when unsequence data is merged. The larger it is, the more IO and CPU cost. The smaller the value, the more the disk is occupied when the unsequence data is too large, the reading will be slower.
Type Int32
Default 0
Effective After restart system
  • enable_stat_monitor
Name enable_stat_monitor
Description Whether to enable background statistics
Type Boolean
Default false
Effective After restart system
  • concurrent_flush_thread
Name concurrent_flush_thread
Description The thread number used to perform the operation when IoTDB writes data in memory to disk. If the value is less than or equal to 0, then the number of CPU cores installed on the machine is used. The default is 0.
Type Int32
Default 0
Effective After restart system
  • tsfile_storage_fs
Name tsfile_storage_fs
Description The storage file system of Tsfile and related data files. Currently LOCAL file system and HDFS are supported.
Type String
Default LOCAL
Effective Only allowed to be modified in first start up
  • core_site_path
Name core_site_path
Description Absolute file path of core-site.xml if Tsfile and related data files are stored in HDFS.
Type String
Default /etc/hadoop/conf/core-site.xml
Effective After restart system
  • hdfs_site_path
Name hdfs_site_path
Description Absolute file path of hdfs-site.xml if Tsfile and related data files are stored in HDFS.
Type String
Default /etc/hadoop/conf/hdfs-site.xml
Effective After restart system
  • hdfs_ip
Name hdfs_ip
Description IP of HDFS if Tsfile and related data files are stored in HDFS. If there are more than one hdfs_ip in configuration, Hadoop HA is used.
Type String
Default localhost
Effective After restart system
  • hdfs_port
Name hdfs_port
Description Port of HDFS if Tsfile and related data files are stored in HDFS
Type String
Default 9000
Effective After restart system
  • dfs_nameservices
Name hdfs_nameservices
Description Nameservices of HDFS HA if using Hadoop HA
Type String
Default hdfsnamespace
Effective After restart system
  • dfs_ha_namenodes
Name hdfs_ha_namenodes
Description Namenodes under DFS nameservices of HDFS HA if using Hadoop HA
Type String
Default nn1,nn2
Effective After restart system
  • dfs_ha_automatic_failover_enabled
Name dfs_ha_automatic_failover_enabled
Description Whether using automatic failover if using Hadoop HA
Type Boolean
Default true
Effective After restart system
  • dfs_client_failover_proxy_provider
Name dfs_client_failover_proxy_provider
Description Proxy provider if using Hadoop HA and enabling automatic failover
Type String
Default org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
Effective After restart system
  • hdfs_use_kerberos
Name hdfs_use_kerberos
Description Whether use kerberos to authenticate hdfs
Type String
Default false
Effective After restart system
  • kerberos_keytab_file_path
Name kerberos_keytab_file_path
Description Full path of kerberos keytab file
Type String
Default /path
Effective After restart system
  • kerberos_principal
Name kerberos_principal
Description Kerberos pricipal
Type String
Default your principal
Effective After restart system
  • authorizer_provider_class
Name authorizer_provider_class
Description the class name of the authorization service
Type String
Default org.apache.iotdb.db.auth.authorizer.LocalFileAuthorizer
Effective After restart system
Other available values org.apache.iotdb.db.auth.authorizer.OpenIdAuthorizer
  • openID_url
Name openID_url
Description the openID server if OpenIdAuthorizer is enabled
Type String (a http url)
Default no
Effective After restart system
  • thrift_max_frame_size
Name thrift_max_frame_size
Description the max bytes in a RPC request/response
Type long
Default 67108864 (should >= 8 * 1024 * 1024)
Effective After restart system

# Automatic Schema Creation and Type Inference

  • enable_auto_create_schema
Name enable_auto_create_schema
Description whether auto create the time series when a non-existed time series data comes
Type true or false
Default true
Effective After restart system
  • default_storage_group_level
Name default_storage_group_level
Description Storage group level when creating schema automatically is enabled. For example, if we receives a data point from root.sg0.d1.s2, we will set root.sg0 as the storage group if storage group level is 1. (root is level 0)
Type integer
Default 1
Effective After restart system
  • boolean_string_infer_type
Name boolean_string_infer_type
Description To which type the values "true" and "false" should be reslved
Type BOOLEAN or TEXT
Default BOOLEAN
Effective After restart system
  • integer_string_infer_type
Name integer_string_infer_type
Description To which type an integer string like "67" in a query should be resolved
Type INT32, INT64, DOUBLE, FLOAT or TEXT
Default DOUBLE
Effective After restart system
  • nan_string_infer_type
Name nan_string_infer_type
Description To which type the value NaN in a query should be resolved
Type DOUBLE, FLOAT or TEXT
Default FLOAT
Effective After restart system
  • floating_string_infer_type
Name floating_string_infer_type
Description To which type a floating number string like "6.7" in a query should be resolved
Type DOUBLE, FLOAT or TEXT
Default FLOAT
Effective After restart system
  • enable_partition
Name enable_partition
Description whether enable data partition. If disabled, all data belongs to partition 0
Type BOOLEAN
Default false
Effective After restart system
  • partition_interval
Name partition_interval
Description time range for partitioning data inside each storage group, the unit is second
Type LONG
Default 604800
Effective After restart system

# Enable GC log

GC log is off by default. For performance tuning, you may want to collect the GC info.

To enable GC log, just add a parameter "printgc" when you start the server.

nohup sbin/start-server.sh printgc >/dev/null 2>&1 &
1

Or

sbin\start-server.bat printgc
1

GC log is stored at IOTDB_HOME/logs/gc.log. There will be at most 10 gc.log.* files and each one can reach to 10MB.

Copyright © 2021 The Apache Software Foundation.
Apache and the Apache feather logo are trademarks of The Apache Software Foundation

Contact us: Join QQ Group 659990460 | Add friend tietouqiao and be invited to Wechat Group
see Join the community for more