Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch the "hooks" branch up with "develop" with a big ol' merge #4927

Open
wants to merge 523 commits into
base: hooks
Choose a base branch
from

Conversation

ximinez
Copy link
Collaborator

@ximinez ximinez commented Feb 23, 2024

High Level Overview of Change

Note that this is not fully functional code! It is being merged into the hooks branch, which is still "in-development".

This branch merges develop into hooks, and makes some fixes / updates to get it partially working.

Results by OS:

  1. It builds on Linux, but a unit test run doesn't finish. Some relatively simple unit tests fail. e.g. AccountSet.
  2. It doesn't build on Windows / MSVC, but it gets a lot further than the original PR did. Notably, it is able to download and set up the wasmedge dependency using Conan. All of the errors appear to be limited to applyHook.cpp, and all appear to be due to parsing / preprocessing errors with the massively complex macros in macros.h
  3. It builds with MacOS clang, but fails to start: dyld[6586]: Library not loaded: /opt/homebrew/opt/llvm/lib/libLLVM.dylib Referenced from: <68C35517-5F51-3521-A8EE-2CE9D9882E28> /Users/ed/.conan/data/wasmedge/0.9.0/_/_/package/72521513b9fee5d76bcfd97cf0256cfa56cb3897/lib/libwasmedge_c.dylib

I have made no significant effort to fix any remaining issues, and I don't intend to, though I will be happy to accept contributions. This is not intended to be working code, just to get the branch caught up so that someone else can get it working.

Context of Change

This is a follow up to #4225, which also only affected that branch.

Type of Change

  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Tests (you added tests for code that already exists, or your new feature included in this PR)

Future Tasks

Significant work and debugging needs to be done to get this "ready for prime time".

Note that this is not fully functional code! It is being merged into the hooks branch, which is still "in-development".

ckeshava and others added 30 commits October 11, 2023 08:45
…#4746)

Update the nix CI runner. This commit does not modify any source code
files. The unix builds were successful, but the binaries were not
uploaded to the internal artifactory. This PR borrows an idea from
@ximinez to attempt to fix this issue.

After successful authentication, the `outcome` variable contains a
string. In the upload step, we are checking if outcome == 'success' as a
prerequisite for uploading the binary.

This commit updates the contents of the `outcome` variable.
The assert is saying that the only reason `pathFinder` would be null is
if the request was aborted (connection dropped, etc.). That's what
`continueCallback()` checks. But that is very clearly not true if you
look at `getPathFinder`, which calls `findPaths`, which can return false
for many reasons.

Fix XRPLF#4744
In Windows, we need to call `python` in order for the `pip` upgrade
command to work.

This changes the GitHub Actions Windows CI job to use the correct
command to upgrade PIP, fixing this error:

```
ERROR: To modify pip, please run the following command:
C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe -m pip install --upgrade pip
```

A future task is to make job run on heavy Windows runners so that it
doesn't take so long.

Context: XRPLF#4596
It might be possible for the server code to indirect through certain
`end()` iterators. While a debug build would catch this problem with
`assert()`s, a release build would crash. If there are problems in this
area in the future, it is best to get a definitive indication of the
nature of the error regardless of whether it's a debug or release build.
To accomplish this, these `assert`s are converted into `LogicError`s
that will produce a reasonable error message when they fire.
Implement native support for W3C DIDs.

Add a new ledger object: `DID`.

Add two new transactions:
1. `DIDSet`: create or update the `DID` object.
2. `DIDDelete`: delete the `DID` object.

This meets the requirements specified in the DID v1.0 specification
currently recommended by the W3C Credentials Community Group.

The DID format for the XRP Ledger conforms to W3C DID standards.
The objects can be created and owned by any XRPL account holder.
The transactions can be integrated by any service, wallet, or application.
Add a new RPC / WS call for `server_definitions`, which returns an
SDK-compatible `definitions.json` (binary enum definitions) generated by
the server. This enables clients/libraries to dynamically work with new
fields and features, such as ones that may become available on side
chains. Clients query `server_definitions` on a node from the network
they want to work with, and immediately know how to speak that node's
binary "language", even if new features are added to it in the future
(as long as there are no new serialized types that the software doesn't
know how to serialize/deserialize).

Example:

```js
> {"command": "server_definitions"}
< {
    "result": {
        "FIELDS": [
            [
                "Generic",
                {
                    "isSerialized": false,
                    "isSigningField": false,
                    "isVLEncoded": false,
                    "nth": 0,
                    "type": "Unknown"
                }
            ],
            [
                "Invalid",
                {
                    "isSerialized": false,
                    "isSigningField": false,
                    "isVLEncoded": false,
                    "nth": -1,
                    "type": "Unknown"
                }
            ],
            [
                "ObjectEndMarker",
                {
                    "isSerialized": false,
                    "isSigningField": true,
                    "isVLEncoded": false,
                    "nth": 1,
                    "type": "STObject"
                }
            ],
        ...
```

Close XRPLF#3657

---------

Co-authored-by: Richard Holland <[email protected]>
The Network ID logic should not be applied to pseudo-transactions.

This allows amendments to enable on a network with an ID > 1024.

Context:
- NetworkID: XRPLF#4370
- Pseudo-transactions: https://xrpl.org/pseudo-transaction-types.html

Fix XRPLF#4736

---------

Co-authored-by: RichardAH <[email protected]>
The unity build speeds up compilation by bundling multiple source files
into one larger file. This reduces Windows CI build time by up to 50%.

As described in XRPLF#4596, the automatic Windows builds take a very long
time. Unity builds are significantly faster - currently about 45 min,
much closer to the typical MacOS (35-40 minutes) and nix (~30 minutes)
run times.

This is intended as a stopgap solution until a more resourced and
reliable runner is available.

No C++ code was changed. This only affects CI.
…F#4733)

Using the "Amount" field in Payment transactions can cause incorrect
interpretation. There continue to be problems from the use of this
field. "Amount" is rarely the correct field to use; instead,
"delivered_amount" (or "DeliveredAmount") should be used.

Rename the "Amount" field to "DeliverMax", a less misleading name. With
api_version: 2, remove the "Amount" field from Payment transactions.

- Input: "DeliverMax" in `tx_json` is an alias for "Amount"
  - sign
  - submit (in sign-and-submit mode)
  - submit_multisigned
  - sign_for
- Output: Add "DeliverMax" where transactions are provided by the API
  - ledger
  - tx
  - tx_history
  - account_tx
  - transaction_entry
  - subscribe (transactions stream)
- Output: Remove "Amount" from API version 2

Fix XRPLF#3484

Fix XRPLF#3902
Update the documentation to describe network health with more nuance as
well as context about related factors.
Remove `tx_history` and `ledger_header` methods from API version 2.

Update `RPC::Handler` to allow for methods (or method implementations)
to be API version specific. This partially resolves XRPLF#4727. We can now
store multiple handlers with the same name, as long as they belong to
different (non-overlapping) API versions. This necessarily impacts the
handler lookup algorithm and its complexity; however, there is no
performance loss on x86_64 architecture, and only minimal performance
loss on arm64 (around 10ns). This design change gives us extra
flexibility evolving the API in the future, including other parts of
XRPLF#4727.

In API version 2, `tx_history` and `ledger_header` are no longer
recognised; if they are called, `rippled` will return error
`unknownCmd`

Resolve XRPLF#3638

Resolve XRPLF#3539
Remove dependency on `<ranges>` header, since it is not implemented by
all compilers which we want to support.

This code change only affects unit tests.

Resolve XRPLF#4787
Introduce the `fixFillOrKill` amendment.

Fix an edge case occurring when an offer with `tfFillOrKill` set (but
without `tfSell` set) fails to cross an offer with a better rate. If
`tfFillOrKill` is set, then the owner must receive the full TakerPays.
Without this amendment, an offer fails if the entire `TakerGets` is not
spent. With this amendment, when `tfSell` is not set, the entire
`TakerGets` does not have to be spent.

For details about OfferCreate, see: https://xrpl.org/offercreate.html

Fix XRPLF#4684

---------

Co-authored-by: Scott Schurr <[email protected]>
With clang 15, an unused-but-set-variable warning was emitted:

PostgresDatabase.cpp:178:14: warning: variable 'expNumResults' set but
not used [-Wunused-but-set-variable]
    uint32_t expNumResults = 1;
The command line API still uses `apiMaximumSupportedVersion`.
The unit test RPCs use `apiMinimumSupportedVersion` if unspecified.

Context:
- XRPLF#4568
- XRPLF#4552
* Remove include <ranges>

* Formatting fix

* Output for subscriptions

* Output from sign, submit etc.

* Output from ledger

* Output from account_tx

* Output from transaction_entry

* Output from tx

* Store close_time_iso in API v2 output

* Add small APIv2 unit test for subscribe

* Add unit test for transaction_entry

* Add unit test for tx

* Remove inLedger from API version 2

* Set ledger_hash and ledger_index

* Move isValidated from RPCHelpers to LedgerMaster

* Store closeTime in LedgerFill

* Time formatting fix

* additional tests for Subscribe unit tests

* Improved comments

* Rename mInLedger to mLedgerIndex

* Minor fixes

* Set ledger_hash on closed ledger, even if not validated

* Update API-CHANGELOG.md

* Add ledger_hash, ledger_index to transaction_entry

* Fix validated and close_time_iso in account_tx

* Fix typos

* Improve getJson for Transaction and STTx

* Minor improvements

* Replace class enum JsonOptions with struct

We may consider turning this into a general-purpose template and using it elsewhere

* simplify the extraction of transactionID from Transaction object

* Remove obsolete comments

* Unconditionally set validated in account_tx output

* Minor improvements

* Minor fixes

---------

Co-authored-by: Chenna Keshava <[email protected]>
* Promote API version 2 to supported

* Switch command line to API version 1

* Fix LedgerRequestRPC test

* Remove obsolete tx_account method

This method is not implemented, the only parts which are removed are related to command-line parsing

* Fix RPCCall test

* Reduce diff size, small test improvements

* Minor fixes

* Support for the mold linker

* [fold] handle case where both mold and gold are installed

* [fold] Use first non-default linker

* Fix TransactionEntry_test

* Fix AccountTx_test

---------

Co-authored-by: seelabs <[email protected]>
* Fix binary output from tx method

* Formatting fix

* Minor test improvement

* Minor test improvements
…uitous proposal changes (XRPLF#4760)

* Optimize the calculation of close time to avoid
impasse and minimize gratuitous proposal changes.

* git apply clang-format.patch

* Review (Howard) fixes.

* Review fix for impasse discovered by John.

* Review fixes (comments) from John.

* Scott S review fixes. Also clang-format.
@vlntb vlntb self-requested a review October 30, 2024 18:06
yinyiqian1 and others added 12 commits November 4, 2024 15:27
Amendment:
- AMMClawback

New Transactions:
- AMMClawback

Modified Transactions:
- AMMCreate
- AMMDeposit
* Add reserve check on AMM Withdraw
* Try AMM max offer if changeSpotPriceQuality() fails
* Checks only Currency or MPT Issuance ID part of the Asset object.
* Resolves temREDUNDANT regression detected in testing.
    Amendment:
    - Credentials
    
    New Transactions:
    - CredentialCreate
    - CredentialAccept
    - CredentialDelete
    
    Modified Transactions:
    - DepositPreauth
    - Payment
    - EscrowFinish
    - PaymentChannelClaim
    - AccountDelete
    
    New Object:
    - Credential

    Modified Object:
    - DepositPreauth
    
    API updates:
    - ledger_entry
    - account_objects
    - ledger_data
    - deposit_authorized
    
    Read full spec: https://github.com/XRPLF/XRPL-Standards/tree/master/XLS-0070d-credentials
* 2.2.2 changed functions acquireAsync and NetworkOPsImp::recvValidation to add an item to a collection under lock, unlock, do some work, then lock again to do remove the item. It will deadlock if an exception is thrown while adding the item - before unlocking.
* Replace ScopedUnlock with scope_unlock.
Copy link
Collaborator

@vlntb vlntb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went through the last 20 synchronizations with develop to check that merges didn't produce any obvious errors. Looks good to me.

* upstream/develop: (505 commits)
  fix: check for valid ammID field in amm_info RPC (5188)
  Set version to 2.3.0-rc2
  fix: include `index` in `server_definitions` RPC (5190)
  Fix ledger_entry crash on invalid credentials request (5189)
  Set version to 2.3.0-rc1
  Replace Uint192 with Hash192 in server_definitions response (5177)
  Fix potential deadlock (5124)
  Introduce Credentials support (XLS-70d): (5103)
  Fix token comparison in Payment (5172)
  Add fixAMMv1_2 amendment (5176)
  Fix unity build (5179)
  Add AMMClawback Transaction (XLS-0073d) (5142)
  Add hubs.xrpkuwait.com to bootstrap (5169)
  docs: Add protobuf dependencies to linux setup instructions (5156)
  fix: reject invalid markers in account_objects RPC calls (5046)
  Update RELEASENOTES.md (5154)
  Introduce MPT support (XLS-33d): (5143)
  Consolidate definitions of fields, objects, transactions, and features (5122)
  Ignore reformat when blaming
  Reformat code with clang-format-18
  ...
intelliot and others added 9 commits November 25, 2024 13:16
`STNumber` lets objects and transactions contain multiple fields for
quantities of XRP, IOU, or MPT without duplicating information about the
"issue" (represented by `STIssue`). It is a straightforward serialization of
the `Number` type that uniformly represents those quantities.

---------

Co-authored-by: John Freeman <[email protected]>
Co-authored-by: Howard Hinnant <[email protected]>
Move tfClawTwoAssets check to preflight and return
error temINVALID_FLAG

---------

Co-authored-by: yinyiqian1 <[email protected]>
* upstream/develop:
  Set version to 2.3.0
  refactor(AMMClawback): move tfClawTwoAssets check (5201)
  Add a new serialized type: STNumber (5121)
* Copy Antithesis SDK version 0.4.0 to directory external/
* Add build option `voidstar` to enable instrumentation with Antithesis SDK
* Define instrumentation macros ASSERT and UNREACHABLE in terms of regular C assert
* Replace asserts with named ASSERT or UNREACHABLE
* Add UNREACHABLE to LogicError
* Document instrumentation macros in CONTRIBUTING.md
* upstream/develop:
  test: Add more test cases for Base58 parser (5174)
  test: Check for some unlikely null dereferences in tests (5004)
  Add Antithesis intrumentation (5042)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.