Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws: add i7ie instance type #561

Open
wants to merge 1 commit into
base: next
Choose a base branch
from

Conversation

syuu1228
Copy link
Contributor

@syuu1228 syuu1228 commented Dec 5, 2024

Adding preset io parameters of i7ie to scylla_cloud_io_setup, and also added i7ie to supported instance type on aws_instance class.

Closes #559

@syuu1228 syuu1228 requested a review from yaronkaikov December 5, 2024 15:28
@yaronkaikov
Copy link
Collaborator

@syuu1228 while you doing that, can you consider also implementing it as a dictionary as described - #544 ?

self.disk_properties["read_iops"] = 352834 * nr_disks
self.disk_properties["read_bandwidth"] = 3422623232 * nr_disks
self.disk_properties["write_iops"] = 119327 * nr_disks
self.disk_properties["write_bandwidth"] = 1526442410 * nr_disks
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do these numbers come from?

Please add to patch changelog.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unlike other clouds like Azure, EC2 does not have IOPS and bandwidth information on their document.
So all preset values are measured by iotune on target instances instead of referenced from the document.
I measured 3 times on each instance types and used average of the result.

Added same description on the changelog.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

previously @xemul had some cavets on how we calculate the numbers for i4i.
I want to make sure we're doing it transparently this time.
Let's have it approved from @xemul as well.

@syuu1228
Copy link
Contributor Author

syuu1228 commented Dec 5, 2024

@syuu1228 while you doing that, can you consider also implementing it as a dictionary as described - #544 ?

I'm working on #544, but I found that it takes few more time to complete implementation so I send this first.
We can import i7ie io parameters to #544 patch later.

@yaronkaikov
Copy link
Collaborator

@syuu1228 please rebase

@syuu1228 syuu1228 force-pushed the aws_i7ie_instance_type branch from 7f6bf1c to 628b575 Compare December 10, 2024 12:39
@syuu1228
Copy link
Contributor Author

@yaronkaikov Rebased with latest master.

@syuu1228 syuu1228 force-pushed the aws_i7ie_instance_type branch from 628b575 to 5dba6f2 Compare December 10, 2024 12:40
@yaronkaikov
Copy link
Collaborator

@avikivity Ok from your POV?

@avikivity
Copy link
Member

yes

@roydahan roydahan requested a review from xemul December 10, 2024 14:03
Copy link
Contributor

@roydahan roydahan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marking as blocked until it gets review for iops and BW numbers.

@roydahan
Copy link
Contributor

@syuu1228 please add the raw output of perftune to the commit message.

Also, can you please elaborate on how you ran iotune?
Was it on our AMI? raw instance of i7ie? without installing scylla and starting it first?
(I'm asking becuase with i4i we noticed differences)

@syuu1228 syuu1228 force-pushed the aws_i7ie_instance_type branch from 5dba6f2 to 07cb8bd Compare December 10, 2024 19:22
Adding preset io parameters of i7ie to scylla_cloud_io_setup,
and also added i7ie to supported instance type on aws_instance class.

All preset values are measured by iotune on target instances.

Here's measurement environment details:
 - Measured on i7ie.* instances with latest version of Ubuntu 24.04 LTS AMI
   (We cannot use Scylla AMI since we do want to measure single drive
   performance)
 - Measured single local SSD w/o RAID0, since we simulate RAID0 performance on
   scylla_cloud_io_setup script from single drive performance
 - Use iotune for the measurement, executed 3 times for each instance size
   and used average of the results
 - Automated measurement by script: https://github.com/syuu1228/ec2_run_script

Here's raw output of iotune:
- i7ie.large (1/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 241 MB/s (deviation 11%)
Measuring sequential read bandwidth: 548 MB/s (deviation 43%)
Measuring random write IOPS: 47148 IOPS (deviation 27%)
Measuring random read IOPS: 58450 IOPS (deviation 24%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.large (2/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 241 MB/s (deviation 11%)
Measuring sequential read bandwidth: 548 MB/s (deviation 43%)
Measuring random write IOPS: 47144 IOPS (deviation 27%)
Measuring random read IOPS: 58447 IOPS (deviation 24%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.large (3/3)
Creating XFS volume for scylla using 1 disk(s): /dev/nvme1n1
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 241 MB/s (deviation 11%)
Measuring sequential read bandwidth: 548 MB/s (deviation 43%)
Measuring random write IOPS: 47145 IOPS (deviation 27%)
Measuring random read IOPS: 58452 IOPS (deviation 24%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.xlarge (1/3)
Creating XFS volume for scylla using 1 disk(s): /dev/nvme1n1
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 482 MB/s (deviation 10%)
Measuring sequential read bandwidth: 1095 MB/s (deviation 43%)
Measuring random write IOPS: 94184 IOPS (deviation 14%)
Measuring random read IOPS: 117261 IOPS (deviation 25%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.xlarge (2/3)
Creating XFS volume for scylla using 1 disk(s): /dev/nvme1n1
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 482 MB/s (deviation 10%)
Measuring sequential read bandwidth: 1095 MB/s (deviation 43%)
Measuring random write IOPS: 94184 IOPS (deviation 14%)
Measuring random read IOPS: 117261 IOPS (deviation 25%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.xlarge (3/3)
Creating XFS volume for scylla using 1 disk(s): /dev/nvme1n1
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 482 MB/s (deviation 10%)
Measuring sequential read bandwidth: 1095 MB/s (deviation 43%)
Measuring random write IOPS: 94184 IOPS (deviation 14%)
Measuring random read IOPS: 117261 IOPS (deviation 25%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.2xlarge (1/3)
Creating XFS volume for scylla using 1 disk(s): /dev/nvme1n1
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 482 MB/s (deviation 10%)
Measuring sequential read bandwidth: 1095 MB/s (deviation 43%)
Measuring random write IOPS: 94166 IOPS (deviation 14%)
Measuring random read IOPS: 117266 IOPS (deviation 25%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.2xlarge (2/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 482 MB/s (deviation 10%)
Measuring sequential read bandwidth: 1094 MB/s (deviation 43%)
Measuring random write IOPS: 94161 IOPS (deviation 14%)
Measuring random read IOPS: 117270 IOPS (deviation 25%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.2xlarge (3/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 482 MB/s (deviation 10%)
Measuring sequential read bandwidth: 1095 MB/s (deviation 43%)
Measuring random write IOPS: 94177 IOPS (deviation 14%)
Measuring random read IOPS: 117266 IOPS (deviation 25%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.3xlarge (1/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 12%)
Measuring sequential read bandwidth: 3263 MB/s (deviation 25%)
Measuring random write IOPS: 119127 IOPS
Measuring random read IOPS: 352843 IOPS (deviation 19%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.3xlarge (2/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 12%)
Measuring sequential read bandwidth: 3263 MB/s (deviation 25%)
Measuring random write IOPS: 118973 IOPS
Measuring random read IOPS: 352844 IOPS (deviation 19%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.3xlarge (3/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 12%)
Measuring sequential read bandwidth: 3265 MB/s (deviation 26%)
Measuring random write IOPS: 119881 IOPS
Measuring random read IOPS: 352817 IOPS (deviation 19%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.6xlarge (1/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 6%)
Measuring sequential read bandwidth: 3273 MB/s (deviation 28%)
Measuring random write IOPS: 118808 IOPS
Measuring random read IOPS: 352813 IOPS (deviation 19%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.6xlarge (2/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 6%)
Measuring sequential read bandwidth: 3266 MB/s (deviation 26%)
Measuring random write IOPS: 119741 IOPS
Measuring random read IOPS: 352837 IOPS (deviation 19%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.6xlarge (3/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 6%)
Measuring sequential read bandwidth: 3269 MB/s (deviation 27%)
Measuring random write IOPS: 119246 IOPS
Measuring random read IOPS: 352822 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.12xlarge (1/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 9%)
Measuring sequential read bandwidth: 3263 MB/s (deviation 25%)
Measuring random write IOPS: 119566 IOPS
Measuring random read IOPS: 352835 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.12xlarge (2/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 9%)
Measuring sequential read bandwidth: 3266 MB/s (deviation 26%)
Measuring random write IOPS: 119214 IOPS
Measuring random read IOPS: 352829 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.12xlarge (3/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 9%)
Measuring sequential read bandwidth: 3265 MB/s (deviation 26%)
Measuring random write IOPS: 118033 IOPS
Measuring random read IOPS: 352832 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.18xlarge (1/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1456 MB/s (deviation 11%)
Measuring sequential read bandwidth: 3267 MB/s (deviation 26%)
Measuring random write IOPS: 119544 IOPS
Measuring random read IOPS: 352825 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.18xlarge (2/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1456 MB/s (deviation 11%)
Measuring sequential read bandwidth: 3266 MB/s (deviation 26%)
Measuring random write IOPS: 119555 IOPS
Measuring random read IOPS: 352816 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.18xlarge (3/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1456 MB/s (deviation 11%)
Measuring sequential read bandwidth: 3269 MB/s (deviation 27%)
Measuring random write IOPS: 119518 IOPS
Measuring random read IOPS: 352815 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.24xlarge (1/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 12%)
Measuring sequential read bandwidth: 3266 MB/s (deviation 26%)
Measuring random write IOPS: 119147 IOPS
Measuring random read IOPS: 352824 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.24xlarge (2/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 12%)
Measuring sequential read bandwidth: 3264 MB/s (deviation 25%)
Measuring random write IOPS: 119154 IOPS
Measuring random read IOPS: 352832 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.24xlarge (3/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 12%)
Measuring sequential read bandwidth: 3266 MB/s (deviation 26%)
Measuring random write IOPS: 119535 IOPS
Measuring random read IOPS: 352826 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.48xlarge (1/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 13%)
Measuring sequential read bandwidth: 3264 MB/s (deviation 25%)
Measuring random write IOPS: 119516 IOPS
Measuring random read IOPS: 352834 IOPS (deviation 18%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.48xlarge (2/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 12%)
Measuring sequential read bandwidth: 3265 MB/s (deviation 26%)
Measuring random write IOPS: 119574 IOPS
Measuring random read IOPS: 352831 IOPS (deviation 19%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

- i7ie.48xlarge (3/3)
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1455 MB/s (deviation 12%)
Measuring sequential read bandwidth: 3269 MB/s (deviation 27%)
Measuring random write IOPS: 119226 IOPS
Measuring random read IOPS: 352815 IOPS (deviation 19%)
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf

Closes scylladb#559
@syuu1228 syuu1228 force-pushed the aws_i7ie_instance_type branch from 07cb8bd to 648873e Compare December 10, 2024 19:24
@syuu1228
Copy link
Contributor Author

@syuu1228 please add the raw output of perftune to the commit message.

Added.

Also, can you please elaborate on how you ran iotune? Was it on our AMI? raw instance of i7ie? without installing scylla and starting it first? (I'm asking becuase with i4i we noticed differences)

Here's measurement environment details:

  • Measured on i7ie.* instances with latest version of Ubuntu 24.04 LTS AMI
    (We cannot use Scylla AMI since we do want to measure single drive performance)
  • Measured single local SSD w/o RAID0, since we simulate RAID0 performance on
    scylla_cloud_io_setup script from single drive performance
  • Use iotune for the measurement, executed 3 times for each instance size
    and used average of the results
  • Automated measurement by script: https://github.com/syuu1228/ec2_run_script

@roydahan
Copy link
Contributor

Thanks for that.
Do we get the same numbers when we run it on Scylla AMI with raid0 on top of all disks?

@syuu1228
Copy link
Contributor Author

Thanks for that. Do we get the same numbers when we run it on Scylla AMI with raid0 on top of all disks?

These are theoretical values, so the actual performance may be lower than these figures depending on CPU performance and other factors.

Since EC2 has a fixed number of Local SSDs, it is possible to measure the performance on the RAID directly and apply the actual performance to the parameters, but this would require re-measuring a significant number of instance types.

@roydahan
Copy link
Contributor

Since EC2 has a fixed number of Local SSDs, it is possible to measure the performance on the RAID directly and apply the actual performance to the parameters, but this would require re-measuring a significant number of instance types.

I'm not asking to re-measure on all instances, but take 2-3 instance type and measure with iotune on the raid0 with Scylla AMI and provide results.
They don't have to be exact but at least in the same area.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AWS i7ie enablement
4 participants