Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add English version of some files & update links in readme.md #424

Merged
merged 9 commits into from
Jun 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ The configuration files are stored in `iot-benchmark/iotdb-1.0/target/iot-benchm
| TDengine | 3.0.1 | TDengine | TDengine-3 |
| PI Archive | 2016 | PIArchive | PIArchive |

* For detailed instructions on using different databases, see [Tested Database Example Instructions](./docs/DifferentTestDatabase.md)
* For detailed instructions on using different databases, see [Tested Database Example Instructions](./docs/DifferentTestDatabase-EN.md)

At the same time, you can also change the BENCHMARK_WORK_MODE parameter to adjust the running mode of iot-benchmark. Currently, the following are supported:

Expand All @@ -245,7 +245,7 @@ At the same time, you can also change the BENCHMARK_WORK_MODE parameter to adjus
| Correctness write mode | verificationWriteMode | Loads data sets from the FILE_PATH path for writing. Currently, IoTDB v1.0 and later versions are supported |
| Correctness query mode | verificationQueryMode | Loads data sets from the FILE_PATH path for comparison with the database. Currently, IoTDB v1.0 and later versions are supported |

* For more mode details, refer to [Overview of different test modes](./docs/DifferentTestMode.md), [Sample configuration of different test modes](./docs//DifferentTestModeConfig.md)
* For more mode details, refer to [Overview of different test modes](./docs/DifferentTestMode-EN.md), [Sample configuration of different test modes](./docs//DifferentTestModeConfig-EN.md)
* For other variable parameter configurations and annotations, please refer to [config.properties](configuration/conf/config.properties), which will not be expanded here.

# 4. Related articles
Expand Down
59 changes: 59 additions & 0 deletions docs/AutoBenchmark-EN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# 1. Automation script

## 1.1. One-click script startup
You can use the `cli-benchmark.sh` script to start IoTDB, monitoring IoTDB Benchmark, and testing IoTDB Benchmark in one click, but please note that the script will clean up **all data** in IoTDB when it starts, so please use it with caution.

First, you need to modify the `IOTDB_HOME` parameter in `cli-benchmark.sh` to the folder where your local IoTDB is located.

Then you can use the script to start the test

```sh
> ./cli-benchmark.sh
```

After the test is completed, you can view the test-related logs in the `logs` folder and the monitoring-related logs in the `server-logs` folder.

## 1.2. Automatic execution of multiple tests

Usually, a single test is meaningless unless it is compared with other test results. Therefore, we provide an interface to execute multiple tests with a single startup.

### 1.2.1. Configure routine

Each line of this file should be a parameter that will change during each test (otherwise it becomes a duplicate test). For example, the "routine" file is:

```
LOOP=10 DEVICE_NUMBER=100 TEST
LOOP=20 DEVICE_NUMBER=50 TEST
LOOP=50 DEVICE_NUMBER=20 TEST
```

Then the test process with 3 LOOP parameters of 10, 20, and 50 is executed in sequence.

> Note:
> You can change multiple parameters in each test using the format of "LOOP=20 DEVICE_NUMBER=10 TEST", and unnecessary space is not allowed. The keyword "TEST" means a new test starts. If you change different parameters, the changed parameters will be retained in the next test.

### 1.2.2. Start the test

After configuring the file routine, you can start the multi-test task by launching the script:

```sh
> ./rep-benchmark.sh
```

Then the test information will be displayed in the terminal.

> Note:
> If you close the terminal or lose the connection with the client machine, the test process will terminate. If the output is piped to a terminal, it is the same as in any other case.

Using this interface usually takes a long time and you may want to execute the test process as a daemon. To do so, you can start the test task as a daemon via a startup script:

```sh
> ./rep-benchmark.sh > /dev/null 2>&1 &
```

In this case, if you want to know what is going on, you can view the log information via the following command:

```sh
> cd ./logs
> tail -f log_info.log
```
10 changes: 10 additions & 0 deletions docs/DeveloperGuide-EN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# 1. Developer Guide
1. All interfaces of iot-benchmark are in the core module.

2. All database tests of iot-benchmark are implemented in various Maven subprojects.

3. If you want to run Benchmark using editors such as IDEA:

-- You can find TestEntrance in the test file directory under each Maven subproject and run the corresponding test.

-- Taking IoTDB 1.0 as an example, you can run `iotdb-1.0/src/main/test/cn/edu/tsinghua/iotdb/benchmark/TestEntrance`
29 changes: 29 additions & 0 deletions docs/DifferentTestDatabase-EN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# 1. Testing other databases with IoTDB Benchmark (../partial)

## 1.1. Testing InfluxDB v1.x
[Quick Guide](../influxdb/README.md)

## 1.2. Testing InfluxDB v2.0
[Quick Guide](../influxdb-2.0/README.md)

## 1.3. Testing Microsoft SQL Server
[Quick Guide](../mssqlserver/README.md)

## 1.4. Testing QuestDB
[Quick Guide](../questdb/README.md)

## 1.5. Testing SQLite
[Quick Guide](../sqlite/README.md)

## 1.6. Testing Victoriametrics
[Quick Guide](../victoriametrics/README.md)

## 1.7. Testing TimeScaleDB
[Quick Guide](../timescaledb/README.md)

## 1.8. Test PI Archive

[Quick Guide](../pi/README.md)

## 1.9. Test TDengine
[Quick Guide](../tdengine/README.md)
121 changes: 121 additions & 0 deletions docs/DifferentTestMode-EN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
## 1. Different operation modes of iot-benchmark

Again, before you start any new test case, you need to confirm whether the configuration information in the configuration file ```config.properties``` meets your expectations.

### 1.1. General test mode: write (single database)

Assume that the workload parameters are:

```
+--------------------------+------------+--------------+-------------+------------+--------------------+--------+
| Measurement/StorageGroup | tag/device | field/sensor | concurrency | batch size | point interval(ms) | loop |
+--------------------------+------------+--------------+-------------+------------+--------------------+--------+
| 10 | 50 | 500 | 20 | 100 | 200 | 10000 |
+--------------------------+------------+--------------+-------------+------------+--------------------+--------+
```

Note: The total number of time series in this configuration is: ```device * sensor = 25,000```, and the number of points in each time series is ```batch size * loop = 20,000```,
The total number of data points is ```deivce * sensor * batch size * loop = 500,000,000```. The space occupied by each data point can be estimated as 16 bytes, so the total size of the raw data is 8G.

Then, the ```config.properties``` file needs to be modified as shown below. Please remove the ```#``` before the corresponding configuration item after completing the modification to ensure that the changes take effect:

```properties
HOST=127.0.0.1
PORT=6667
DB_SWITCH=IoTDB-100-SESSION_BY_TABLET
BENCHMARK_WORK_MODE=testWithDefaultPath
OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
GROUP_NUMBER=10
DEVICE_NUMBER=50
SENSOR_NUMBER=500
CLIENT_NUMBER=20
BATCH_SIZE_PER_WRITE=100
POINT_STEP=200
LOOP=10000
```

If you want to write data in a disorderly manner, you need to modify the following properties of the `config.properties` file:

```
# Whether to write out of order
IS_OUT_OF_ORDER=true
# Out of order write mode, currently there are 2 types
# POISSON Out of order mode according to Poisson distribution
# BATCH Batch insert out of order mode
OUT_OF_ORDER_MODE=BATCH
# The proportion of data written out of order
OUT_OF_ORDER_RATIO=0.5
# Is it equal time stamp
IS_REGULAR_FREQUENCY=true
# Expectation and variance of Poisson distribution
LAMBDA=2200.0
# Maximum value of random number of Poisson distribution model
MAX_K=170000
```

Some basic modification operations of parameters have been introduced here, and the relevant settings will be omitted in the following part. If necessary, please enter the following page to view.
[Testconfigurations](https://github.com/supersshhhh/iot-benchmark/blob/patch-1/Testconfigurations.md)

## 2. General test mode: query (single database, no system records)

In addition to writing data, the general test mode can also query data.

## 3. General test mode: read-write mixed mode (single database)

General test mode can support users to perform read-write mixed tests. It should be noted that the timestamps of read-write mixed in this scenario all start from the **write start time**.

## 4. General test mode: read-write mixed mode (single database, query the most recently written data)

General test mode can support users to perform read-write mixed tests (query the most recently written data). It should be noted that the query time range in this scenario is the data adjacent to the left of the current maximum write timestamp.

## 5. General test mode: use system records (single database)

IoTDB Benchmark supports you to use the database to store system data during the test. Currently, it supports the use of CSV records.

## 6. Conventional test mode: test process persistence (single database)

For subsequent analysis, iot-benchmark can store test information in the database (if you do not want to store test data, set ```TEST_DATA_PERSISTENCE=None```)

## 7. Generate data mode

In order to generate reusable data sets, iot-benchmark provides a mode for generating data sets, generating data sets to FILE_PATH for subsequent use in correctness write mode and correctness query mode.

## 8. Correctness write mode (single database, external data set)

In order to verify the correctness of data set writing, you can use this mode to write the data set generated in the generate data mode. Currently, this mode only supports IoTDB v1.0 and later versions and InfluxDB v1.x

## 9. Correctness single-point query mode (single database, external data set)

Before running this mode, you need to use the correctness write mode to write data to the database. To verify the correctness of the data set written, you can use this mode to query the data set written to the database. Currently, this mode only supports IoTDB v1.0 and InfluxDB v1.x.

## 10. Dual database mode

To complete the correctness verification more conveniently and quickly, iot-benchmark also supports dual database mode.

1. For all the test scenarios mentioned above, unless otherwise specified, dual databases are supported. Please **start the test** in the `verification` project.

2. For the relevant test scenarios for correctness verification below, they must be run in dual database mode, and currently only IoTDB v1.0 and newer versions and timescaledb are supported.

## 11. General test mode: write (dual database)

In order to perform the correctness verification below, you first need to write the data to two databases.

## 12. Correctness single point query mode (dual database comparison)

In order to more efficiently verify the correctness of database data, iot-benchmark provides correctness verification by comparing the data between two databases. Note: Before performing this test, please use the general test mode of writing (dual databases) to complete the database writing. Currently, it is recommended to use the JDBC method.

## 13. Correctness function query mode (dual database comparison)

In order to more efficiently verify the correctness of database queries, iot-benchmark provides correctness verification by comparing the differences in data query results between two databases.

Note:

1. Before performing this test, please use the general test mode of writing (dual databases) to complete the database writing.

2. The value of LOOP cannot be too large, satisfying: LOOP(query) * QUERY_INTERVAL(query) * DEVICE_NUMBER(write) <= LOOP(write) * POINT_STEP(write)

## 14. Further explanation of correctness verification

1. Currently, the correctness verification part only supports IoTDB v1.0 and later versions and TimeScaleDB

2. [Quick Guide](../verification/README.md)
Loading
Loading