Skip to content

Increase max open files parameters in rocksdb up to 10K - #6607

Merged
EdvardD merged 13 commits into
masterfrom
increase_max_open_files_up_to_10_000
Apr 14, 2022
Merged

Increase max open files parameters in rocksdb up to 10K#6607
EdvardD merged 13 commits into
masterfrom
increase_max_open_files_up_to_10_000

Conversation

@EdvardD

@EdvardD EdvardD commented Apr 14, 2022

Copy link
Copy Markdown
Contributor

Increase the default max_open_files from 512 up to 10k;
Set ulimit for the number of files from neard to 11025.

@EdvardD
EdvardD requested a review from a team as a code owner April 14, 2022 12:18
@EdvardD
EdvardD requested a review from mina86 April 14, 2022 12:18

@mina86 mina86 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to also update recompress_storage command so that it sets the limit to ‘2*max_open_files + 1000’ or something along those lines.

I should have noticed this in previous PR, but max_open_files should probably be u32 rather than i32.

Comment thread core/store/src/db.rs Outdated
Comment thread core/store/src/db.rs Outdated
@EdvardD

EdvardD commented Apr 14, 2022

Copy link
Copy Markdown
Contributor Author

You need to also update recompress_storage command so that it sets the limit to ‘2*max_open_files + 1000’ or something along those lines.

I should have noticed this in previous PR, but max_open_files should probably be u32 rather than i32.

This doesn't prevent recompress_storage from setting their own 3 * 512 limit.

@EdvardD

EdvardD commented Apr 14, 2022

Copy link
Copy Markdown
Contributor Author

@EdvardD EdvardD closed this Apr 14, 2022
@EdvardD EdvardD reopened this Apr 14, 2022
@EdvardD

EdvardD commented Apr 14, 2022

Copy link
Copy Markdown
Contributor Author

I should have noticed this in previous PR, but max_open_files should probably be u32 rather than i32.
It was u32 until I noticed that rocksdb has it as i32 so I made it i32 to be consistent. It doesn't seem to be a blocker for this change.

@EdvardD
EdvardD requested a review from mina86 April 14, 2022 15:48
Comment thread core/store/src/db.rs Outdated
Comment thread core/store/src/db.rs Outdated
Comment thread core/store/src/db.rs Outdated
@mina86

mina86 commented Apr 14, 2022

Copy link
Copy Markdown
Contributor

OK, #6612 is now in so this is just a matter of minor stylistic changes.

Comment thread core/store/src/db.rs Outdated
@EdvardD
EdvardD requested a review from mina86 April 14, 2022 16:43
Comment thread CHANGELOG.md Outdated
EdvardD and others added 4 commits April 14, 2022 18:52
Co-authored-by: Michal Nazarewicz <mina86@mina86.com>
@EdvardD
EdvardD merged commit 17a5be6 into master Apr 14, 2022
@EdvardD
EdvardD deleted the increase_max_open_files_up_to_10_000 branch April 14, 2022 18:09
pompon0 pushed a commit that referenced this pull request Apr 15, 2022
* Increase the default max_open_files from 512 up to 10k;
* Set ulimit for the number of files from neard to 11025.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants