Skip to main content

Auto create metadata

About 2 min

Auto create metadata

Automatically creating schema means creating time series based on the characteristics of written data in case time series haven't defined by users themselves.
This function can not only solve the problem that entities and measurements are difficult to predict and model in advance under massive time series scenarios,
but also provide users with an out-of-the-box writing experience.

Auto create database metadata

  • enable_auto_create_schema
Nameenable_auto_create_schema
Descriptionwhether creating schema automatically is enabled
Typeboolean
Defaulttrue
EffectiveAfter restarting system
  • default_storage_group_level
Namedefault_storage_group_level
DescriptionSpecify which level database is in the time series, the default level is 1 (root is on level 0)
Typeint
Default1
EffectiveOnly allowed to be modified in first start up

Illustrated as the following figure:

  • When default_storage_group_level=1, root.turbine1 and root.turbine2 will be created as database.

  • When default_storage_group_level=2, root.turbine1.d1, root.turbine1.d2, root.turbine2.d1 and root.turbine2.d2 will be created as database.

auto create database example

Auto create time series metadata(specify data type in the frontend)

  • Users should specify data type when writing:

    • insertTablet method in Session module.
    • insert method using TSDataType in Session module.
      public void insertRecord(String deviceId, long time, List<String> measurements, List<TSDataType> types, Object... values);
      public void insertRecords(List<String> deviceIds, List<Long> times, List<List<String>> measurementsList, List<List<TSDataType>> typesList, List<List<Object>> valuesList);
      
    • ......
  • Efficient, time series are auto created when inserting data.

Auto create time series metadata(infer data type in the backend)

  • Just pass string, and the database will infer the data type:

    • insert command in CLI module.
    • insert method without using TSDataType in Session module.
      public void insertRecord(String deviceId, long time, List<String> measurements, List<TSDataType> types, List<Object> values);
      public void insertRecords(List<String> deviceIds, List<Long> times, List<List<String>> measurementsList, List<List<String>> valuesList);
      
    • ......
  • Since type inference will increase the writing time, the efficiency of auto creating time series metadata through type inference is lower than that of auto creating time series metadata through specifying data type. We recommend users choose specifying data type in the frontend when possible.

Type inference

Data(String Format)Format Typeiotdb-datanode.propertiesDefault
truebooleanboolean_string_infer_typeBOOLEAN
1integerinteger_string_infer_typeFLOAT
17000000(integer > 2^24)integerlong_string_infer_typeDOUBLE
1.2floatingfloating_string_infer_typeFLOAT
NaNnannan_string_infer_typeDOUBLE
'I am text'textxx
  • Data types can be configured as BOOLEAN, INT32, INT64, FLOAT, DOUBLE, TEXT.

  • long_string_infer_type is used to avoid precision loss caused by using integer_string_infer_type=FLOAT to infer num > 2^24.

Encoding Type

Data Typeiotdb-datanode.propertiesDefault
BOOLEANdefault_boolean_encodingRLE
INT32default_int32_encodingRLE
INT64default_int64_encodingRLE
FLOATdefault_float_encodingGORILLA
DOUBLEdefault_double_encodingGORILLA
TEXTdefault_text_encodingPLAIN
  • Encoding types can be configured as PLAIN, RLE, TS_2DIFF, GORILLA, DICTIONARY.

  • The corresponding relationship between data types and encoding types is detailed in Encoding.

Copyright © 2024 The Apache Software Foundation.
Apache and the Apache feather logo are trademarks of The Apache Software Foundation

Have a question? Connect with us on QQ, WeChat, or Slack. Join the community now.