Skip to content

feat: allow comments in config.json, genesis.json, node_key.json and validator_key.json - #8423

Merged
ppca merged 6 commits into
near:masterfrom
ppca:xiangyiz/ND-271
Feb 1, 2023
Merged

feat: allow comments in config.json, genesis.json, node_key.json and validator_key.json#8423
ppca merged 6 commits into
near:masterfrom
ppca:xiangyiz/ND-271

Conversation

@ppca

@ppca ppca commented Jan 23, 2023

Copy link
Copy Markdown
Contributor

This is useful for dynamic config.
For this to work for node start, nearup also needs to be updated.

@ppca
ppca requested a review from nikurt January 23, 2023 23:35

@nikurt nikurt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also allow comments in node_key.json, i.e. https://github.com/near/nearcore/blob/master/nearcore/src/config.rs#L1293 ?
Should we also allow comments in validator_key.json, i.e. https://github.com/near/nearcore/blob/master/core/primitives/src/validator_signer.rs#L163 ?

nearup

I don't think that updating nearup is needed. nearup is not the recommended way of running nodes. If comments are not allowed in those conditions, that is acceptable.

let file = File::open(path).with_context(|| "Could not open genesis config file.")?;
let reader = BufReader::new(file);
let genesis_config: GenesisConfig = serde_json::from_reader(reader)
// Strip the comments from the input (use `as_bytes()` to get a `Read`).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Strip the comments from the input (use `as_bytes()` to get a `Read`).
// Strip the comments from the input.
// `as_bytes()` returns a `Read`.

Comment thread nearcore/src/dyn_config.rs Outdated
tracing::info!(target: "neard", config=?config, "Changing the config {path:?}.");
return Ok(Some(config));
Ok(config_str) => {
// Strip the comments from the input (use `as_bytes()` to get a `Read`).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

@ppca ppca changed the title Use json_comments crate to skip config file comments feat: allow comments in config.json, node_key.json and validator.json Jan 31, 2023
@ppca ppca changed the title feat: allow comments in config.json, node_key.json and validator.json feat: allow comments in config.json, genesis.json, node_key.json and validator.json Jan 31, 2023
@ppca

ppca commented Jan 31, 2023

Copy link
Copy Markdown
Contributor Author

Should we also allow comments in node_key.json, validator_key.json
Added.

I've tested for dynamic config field expected_shutdown in localnet, the following comments are allowed:
Screenshot 2023-01-30 at 6 20 24 PM

The strip comments for the other json's should also work but not sure how we can test them?

@ppca
ppca marked this pull request as ready for review January 31, 2023 02:21
@ppca
ppca requested a review from a team as a code owner January 31, 2023 02:21
@nikurt

nikurt commented Jan 31, 2023

Copy link
Copy Markdown
Contributor

The strip comments for the other json's should also work but not sure how we can test them?

Add comments to node_key.json and validator_key.json. If the node starts then comments are indeed stripped.

@@ -261,18 +261,25 @@ impl GenesisConfig {
///

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a comment that it's not JSON string, but JSON-with-comments string.

Comment thread nearcore/src/config.rs
@@ -395,11 +395,12 @@ impl Default for Config {

impl Config {
pub fn from_file(path: &Path) -> anyhow::Result<Self> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment that the file can be JSON-with-comments.

Comment thread utils/config/Cargo.toml
@@ -0,0 +1,11 @@
[package]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add this in the global Cargo.toml as a workspace member. Around line 60.

Comment thread utils/config/src/lib.rs

use json_comments::StripComments;

pub fn strip_comments_from_json_str(json_str: &String) -> std::io::Result<String> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea to move all the code that deals with json_comments to a separate module.
👍

@ppca
ppca force-pushed the xiangyiz/ND-271 branch 2 times, most recently from 8ef8e2d to 693e452 Compare February 1, 2023 16:11
@ppca
ppca merged commit fd4b385 into near:master Feb 1, 2023
@ppca
ppca deleted the xiangyiz/ND-271 branch February 1, 2023 17:31
@ppca ppca changed the title feat: allow comments in config.json, genesis.json, node_key.json and validator.json feat: allow comments in config.json, genesis.json, node_key.json and validator_key.json Feb 1, 2023
@ppca ppca changed the title feat: allow comments in config.json, genesis.json, node_key.json and validator_key.json feat: allow comments in config.json, genesis.json, node_key.json Feb 1, 2023
@ppca ppca changed the title feat: allow comments in config.json, genesis.json, node_key.json feat: allow comments in config.json, genesis.json, node_key.json and validate_key.json Feb 1, 2023
@ppca ppca changed the title feat: allow comments in config.json, genesis.json, node_key.json and validate_key.json feat: allow comments in config.json, genesis.json, node_key.json and validator_key.json Feb 1, 2023
nikurt pushed a commit that referenced this pull request Feb 6, 2023
…validator.json (#8423)

* Use json-comments-rs crate to skip config file comments

* strip comments out from json for genesis_config.json, config.json, validator_key.json and node_key.json

* strip comments functions sit in near-config-utils (utils/config)
@iho

iho commented Jul 11, 2023

Copy link
Copy Markdown
Contributor

What motivation for having comments in JSON? Can I remove this feature? With this feature, we have higher startup time.

#9127 (comment)

@nikurt

nikurt commented Jul 12, 2023

Copy link
Copy Markdown
Contributor

The PR description could have been more explicit about it. It greatly speeds up development of features involving config changes.
No, please don't remove this feature.
Responded on the linked comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants