Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataset set postmatch 5576 v12 #12057

Closed

Conversation

catenacyber
Copy link
Contributor

Link to ticket: https://redmine.openinfosecfoundation.org/issues/
https://redmine.openinfosecfoundation.org/issues/5576
https://redmine.openinfosecfoundation.org/issues/7326

Describe changes:

SV_BRANCH=OISF/suricata-verify#2093

#12053 with compiler warning fixes and clean git history

This shows the POC to have feedback
Feedback is excepted on the design :

  • detect: postmatch can run AppLayerTxMatch callbacks (in its own commit)
  • usage of delay_postmatch: buffers using dataset/set are put at the tail of inspections whatever their progress
  • usage of DETECT_ENGINE_INSPECT_SIG_MATCH_MORE_BUF: dataset may return this new case to tell that ok we match on a multi-buffer, but we still want to try all occurrences of a multi-buffer (instead of returning a simple early match)
  • How do you like the git history/commits ?

Will be useful for dataset, when it needs to find a transaction
buffer again.
The set operation of dataset keyword was done even if signature
did not fully match, which is not the expected behavior.
We want dataset to behave like flowbits for instance.

This patch changes the behavior of the dataset keyword to do a
match and a post match for the set operation.
The postmatch retrieves the data, using the list identifier
associated to the buffer for this signature.

This avoids to store the buffer(s), when we do not have a
dedicated storage (per signature and per tx) that can own
and clean arbitrary buffers over multiple packets, in
the case the transaction spans over multiple packets
with different tx progresses for instance.
If detection runs on one packet, the InspectionBuffer are
cached and fast to get.
The most expensive case if for multi buffers, where we
need to run detection again, to see which occurences
match all payload keywords and should be added in the dataset.

Ticket: OISF#5576
Copy link

codecov bot commented Oct 29, 2024

Codecov Report

Attention: Patch coverage is 88.37209% with 20 lines in your changes missing coverage. Please review.

Project coverage is 83.41%. Comparing base (3a7eef8) to head (3ff76a5).
Report is 43 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #12057      +/-   ##
==========================================
- Coverage   83.42%   83.41%   -0.02%     
==========================================
  Files         910      910              
  Lines      257642   257773     +131     
==========================================
+ Hits       214949   215013      +64     
- Misses      42693    42760      +67     
Flag Coverage Δ
fuzzcorpus 61.58% <76.74%> (-0.06%) ⬇️
livemode 19.47% <44.18%> (+0.06%) ⬆️
pcap 44.42% <19.76%> (-0.07%) ⬇️
suricata-verify 62.79% <88.37%> (+0.03%) ⬆️
unittests 59.34% <19.18%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline 23203

@catenacyber
Copy link
Contributor Author

Rebased in #12126

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants