Skip to content

stream: refactor clearBuffer and writableState#45682

Closed
anonrig wants to merge 2 commits intonodejs:mainfrom
anonrig:refactor/writable
Closed

stream: refactor clearBuffer and writableState#45682
anonrig wants to merge 2 commits intonodejs:mainfrom
anonrig:refactor/writable

Conversation

@anonrig
Copy link
Member

@anonrig anonrig commented Nov 29, 2022 •
edited
Loading

This pull request refactors clearBuffer function and removes the unnecessary typeof isDuplex check on WritableState class.

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Nov 29, 2022

Review requested:

  • @nodejs/streams

@nodejs-github-bot nodejs-github-bot added the needs-ci PRs that need a full CI run. label Nov 29, 2022
@anonrig anonrig marked this pull request as ready for review November 29, 2022 19:51
@anonrig anonrig added the performance Issues and PRs related to the performance of Node.js. label Nov 29, 2022
@anonrig anonrig requested a review from mcollina November 29, 2022 19:54
@anonrig anonrig added the stream Issues and PRs related to the stream subsystem. label Nov 29, 2022
Uzlopak
Uzlopak reviewed Nov 29, 2022
View reviewed changes
@mscdex
Copy link
Contributor

mscdex commented Nov 29, 2022

Doesn't seem to be reproducible. A re-run of the benchmarks shows:

streams/writable-manywrites.js len=1024 callback='no' writev='no' sync='no' n=2000000              *     -1.43 %       ±1.12% ±1.49% ±1.95%
streams/writable-manywrites.js len=1024 callback='no' writev='no' sync='yes' n=2000000                   -1.73 %       ±3.29% ±4.38% ±5.71%
streams/writable-manywrites.js len=1024 callback='no' writev='yes' sync='no' n=2000000                    1.52 %       ±2.90% ±3.86% ±5.02%
streams/writable-manywrites.js len=1024 callback='no' writev='yes' sync='yes' n=2000000                   2.19 %       ±3.46% ±4.60% ±6.00%
streams/writable-manywrites.js len=1024 callback='yes' writev='no' sync='no' n=2000000             *     -2.91 %       ±2.53% ±3.38% ±4.44%
streams/writable-manywrites.js len=1024 callback='yes' writev='no' sync='yes' n=2000000                  -0.08 %       ±3.22% ±4.29% ±5.58%
streams/writable-manywrites.js len=1024 callback='yes' writev='yes' sync='no' n=2000000                   2.97 %       ±5.28% ±7.08% ±9.34%
streams/writable-manywrites.js len=1024 callback='yes' writev='yes' sync='yes' n=2000000                 -2.02 %       ±4.66% ±6.21% ±8.11%
streams/writable-manywrites.js len=32768 callback='no' writev='no' sync='no' n=2000000                   -0.63 %       ±1.29% ±1.71% ±2.23%
streams/writable-manywrites.js len=32768 callback='no' writev='no' sync='yes' n=2000000                   2.60 %       ±3.30% ±4.44% ±5.88%
streams/writable-manywrites.js len=32768 callback='no' writev='yes' sync='no' n=2000000                   1.84 %       ±2.69% ±3.61% ±4.76%
streams/writable-manywrites.js len=32768 callback='no' writev='yes' sync='yes' n=2000000                  1.35 %       ±4.10% ±5.50% ±7.26%
streams/writable-manywrites.js len=32768 callback='yes' writev='no' sync='no' n=2000000                   0.15 %       ±1.46% ±1.94% ±2.53%
streams/writable-manywrites.js len=32768 callback='yes' writev='no' sync='yes' n=2000000                  0.55 %       ±1.83% ±2.44% ±3.18%
streams/writable-manywrites.js len=32768 callback='yes' writev='yes' sync='no' n=2000000                 -0.10 %       ±2.19% ±2.91% ±3.79%
streams/writable-manywrites.js len=32768 callback='yes' writev='yes' sync='yes' n=2000000                 2.10 %       ±4.64% ±6.22% ±8.20%

@anonrig
Copy link
Member Author

anonrig commented Nov 29, 2022

Doesn't seem to be reproducible. A re-run of the benchmarks shows:

Thanks for the update @mscdex. Do you have any particular reason for this flakiness? I've added a new commit and re-run the benchmark CI. If it's still the same, I'll update the description and remove the references to the benchmark results.

@mscdex
Copy link
Contributor

mscdex commented Nov 29, 2022

Do you have any particular reason for this flakiness?

No, but it's always best to re-run the benchmarks several times to ensure consistent results.

@anonrig anonrig changed the title stream: refactor clearBuffer stream: refactor clearBuffer and writableState Nov 29, 2022
@anonrig anonrig added the commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. label Nov 29, 2022
@mscdex mscdex added the needs-citgm PRs that need a CITGM CI run. label Nov 30, 2022
@anonrig anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Nov 30, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 30, 2022
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Nov 30, 2022

CI: https://ci.nodejs.org/job/node-test-pull-request/48240/

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Nov 30, 2022

CI: https://ci.nodejs.org/job/node-test-pull-request/48241/

ronag
ronag requested changes Nov 30, 2022
View reviewed changes
lib/internal/streams/writable.js
const len = objectMode ? 1 : chunk.length;
doWrite(stream, state, false, len, chunk, encoding, callback);
} while (i < buffered.length && !state.writing);
}
Copy link
Member

@ronag ronag Nov 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unnecessary change

ronag
ronag requested changes Nov 30, 2022
View reviewed changes
lib/internal/streams/writable.js
return;
}

state.bufferProcessing = true;
Copy link
Member

@ronag ronag Nov 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why this is better?

Copy link
Member Author

@anonrig anonrig Nov 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary length operation was done before state.bufferProcessing assignment is done. This change eliminates that.

Copy link
Member

@ronag ronag Nov 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't see it. This is actually slower for the case where bufferedLength is 0... since it unnecessarily sets and unsets bufferProcessing.

@anonrig anonrig closed this Dec 1, 2022
This was referenced Dec 7, 2022
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@santigimeno santigimeno santigimeno left review comments

@ronag ronag ronag requested changes

@mcollina mcollina Awaiting requested review from mcollina

+1 more reviewer

@Uzlopak Uzlopak Uzlopak left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. needs-ci PRs that need a full CI run. needs-citgm PRs that need a CITGM CI run. performance Issues and PRs related to the performance of Node.js. stream Issues and PRs related to the stream subsystem.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@anonrig @nodejs-github-bot @mscdex @mcollina @santigimeno @ronag @Uzlopak