Basic Functional Test (REC-31)

[REC-40] Storage rpm test Created: 26/Aug/19  Updated: 04/Oct/19

Status: To Do
Project: Radio Edge Cloud
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Sub-task Priority: Medium
Reporter: Deepak Kataria Assignee: Indumathi Buddi
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Sequential read/write IOPS

Random read/write IOPS

Throughput for storage read/write operations

Latency for storage read/write operations

Consider using Ceph Benchmark Tool (CBT). CBT is a Python tool for building a Ceph cluster and running benchmarks against the cluster. It also provides the test harness for automating various load test utilities such as RADOS bench, FIO, COSBench and MySQL sysbench.

FIO can be used to measure sequential and random throughput performance and latency



 Comments   
Comment by Deepak Kataria [ 04/Oct/19 ]

Here is what we plan to use:

  • Kubernetes Pod
  • Persistent Volume (PV) backed by a Ceph RBD image
  • FIO (a block testing tool) on the mounted location of the above PV
  • The parameters of the FIO tests (e.g. block size, threads, etc) would be inputs to the test.   Alternatively, we could set up few sample parameter
  • sets to mimic a few traffic profiles and run those.
  • The FIO results could then be compared baseline numbers to determine whether it is passes.
  • FIO will be used to measure sequential and random throughput performance and latency

 

Comment by Krisztián Lengyel [ 04/Oct/19 ]

Elasticsearch uses the host path /var/log/elasticsearch to store its data.

 

About E2 Termination, I believe that's out of REC scope.

Comment by Deepak Kataria [ 02/Oct/19 ]

Logging: Each component writes log files (sequential write) and REC has a fluentd configured to send the data to Elastic Search. How does Elastic Search uses storage internally (sequential/random)?

Redis DB: in memory only, does not use disk at all

Message archiving: There is a plan that the E2 Termination writes all the messages that pass through it to disk (sequential write) and periodically these files would be copied either to another local server or send to Research Data Lake using Data Router.

So, except for whatever Elastic Search does internally, all the writes and reads would be sequential and each data item would be written and read one time (50/50).

As far as throughput versus latency, throughput would be more important. We can test IOPS for 4KB average size. Throughput in MB/s=Average IO size x IOPS. Each IO request will take time to complete, this is the average latency, measured in milliseconds (ms).

  1. One IOPS test case can be: 4KB block size, sequential read/write (50/50)
  2. Second IOPS test case can be: 4KB block size, random read/write (70/30)
Comment by Krisztián Lengyel [ 27/Sep/19 ]

This seems like a pure performance test case. Are there any targets for the measured properties?

Generated at Sat Feb 10 06:04:37 UTC 2024 using Jira 9.4.5#940005-sha1:e3094934eac4fd8653cf39da58f39364fb9cc7c1.