Processes scaling better than domainsI made a tree-sitter plugin for
+dune files. It is available on
+GitHub.
Tree-sitter is a parsing system that can be used in text editors.
+Dune is a build system for OCaml projects.
+Its configuration language lives in dune files which use a s-expression
+syntax.
This makes highlighting challenging: the lexing part of the language is very +simple (atoms, strings, parentheses), but it is not enough to make a good +highlighter.
+In the following example, with-stdout-to and echo are “actions” that we
+could highlight in a special way, but these names can also appear in places
+where they are not interpreted as actions, and doing so would be confusing (for
+example, we could write to a file named echo instead of foo.txt.
(rule
+ (action
+ (with-stdout-to
+ foo.txt
+ (echo "testing"))))Tree-sitter solves this, because it creates an actual parser that goes beyond +lexing.
+In this example, I created grammar rules that parse the contents of (action ...) as an action, recognizing the various constructs of this DSL.
The output of the parser is this syntax tree with location information (for +some reason, line numbers start at 0 which is normal and unusual at the same +time).
+(source_file [0, 0] - [5, 0]
+ (stanza [0, 0] - [4, 22]
+ (stanza_name [0, 1] - [0, 5])
+ (field_name [1, 2] - [1, 8])
+ (action [2, 2] - [4, 20]
+ (action_name [2, 3] - [2, 17])
+ (file_name_target [3, 3] - [3, 10]
+ (file_name [3, 3] - [3, 10]))
+ (action [4, 3] - [4, 19]
+ (action_name [4, 4] - [4, 8])
+ (quoted_string [4, 9] - [4, 18])))))The various strings are annotated with their type: we have stanza names
+(rule), field names (action), action names (with-stdout-to, echo), file
+names (foo.txt), and plain strings ("testing").
By itself, that is not useful, but it’s possible to write queries to make +this syntax tree do interesting stuff.
+The first one is highlighting: we can set styles for various “patterns” (in +practice, I only used node names) by defining queries:
+(stanza_name) @function
+(field_name) @property
+(quoted_string) @string
+(multiline_string) @string
+(action_name) @keywordThe parts with @ map to “highlight groups” used in text editors.
Another type of query is called “injections”. It is used to link different
+types of grammars together. For example, dune files can start with a special
+comment that indicates that the rest of the file is an OCaml program. In that
+case, the parser emits a single ocaml_syntax node and the following injection
+indicates that this file should be parsed using an OCaml parser:
((ocaml_syntax) @injection.content
+ (#set! injection.language "ocaml"))Another use case for this is system actions: these strings in dune files
+could be interpreted using a shell parser.
In the other direction, it is possible to inject dune files into another
+document. For example, a markdown parser can use injections to highlight code
+blocks.
I’m happy to have explored this technology. The toolchain seemed complex at +first: there’s a compiler which seems to be a mix of node and rust, which +generates C, which is compiled into a dynamically loaded library; but this is +actually pretty well integrated in nix and neovim to the details are made +invisible.
+The testing mechanism is similar to the cram tests we use in Dune, but I was a
+bit confused with the colors at first: when the output of a test changes, Dune
+considers that the new output is a + in the diff, and highlights it in green;
+while tree-sitter considers that the “expected output” is green.
There are many ways to improve this prototype: either by adding queries (it’s
+possible to define text objects, folding expressions, etc), or by improving
+coverage for dune files (in most cases, the parser uses a s-expression
+fallback). I’m also curious to see if it’s possible to use this parser to
+provide a completion source. Since the strings are tagged with their type (are
+we expecting a library name, a module name, etc), I think we could use that to
+provide context-specific completions, but that’s probably difficult to do.
Thanks teej for the initial idea and the useful +resources.
diff --git a/data/planet/emilpriver/why-i-like-ocaml.md b/data/planet/emilpriver/why-i-like-ocaml.md new file mode 100644 index 0000000000..a92cddd8f1 --- /dev/null +++ b/data/planet/emilpriver/why-i-like-ocaml.md @@ -0,0 +1,374 @@ +--- +title: Why I Like Ocaml +description: I like OCaml and this is why +url: https://priver.dev/blog/ocaml/why-i-like-ocaml/ +date: 2024-07-21T12:10:55-00:00 +preview_image: https://priver.dev/images/ocaml/ocaml-cover.png +authors: +- "Emil Priv\xE9r" +source: +--- + +According to my Linkedin profile, I have been writing code for a company for almost 6 years. During this time, I have worked on PHP and Wordpress projects, built e-commerce websites using NextJS and JavaScript, written small backends in Python with Django/Flask/Fastapi, and developed fintech systems in GO, among other things. I have come to realize that I value a good type system and prefer writing code in a more functional way rather than using object-oriented programming. For example, in GO, I prefer passing in arguments rather than creating a struct method. This is why I will be discussing OCaml in this article.
If you are not familiar with the language OCaml or need a brief overview of it, I recommend reading my post OCaml introduction before continuing with this post. It will help you better understand the topic I am discussing.
+Almost every time I ask someone what they like about OCaml, they often say “oh, the type system is really nice” or “I really like the Hindley-Milner type system.” When I ask new OCaml developers what they like about the language, they often say “This type system is really nice, Typescript’s type system is actually quite garbage.” I am not surprised that these people say this, as I agree 100%. I really enjoy the Hindley-Milner type system and I think this is also the biggest reason why I write in this language. A good type system can make a huge difference for your developer experience.
+For those who may not be familiar with the Hindley-Milner type system, it can be described as a system where you write a piece of program with strict types, but you are not required to explicitly state the types. Instead, the type is inferred based on how the variable is used. +Let’s look at some code to demonstrate what I mean. In GO, you would be required to define the type of the arguments:
+
+ |
+
+ |
However, in OCaml, you don’t need to specify the type:
+
+ |
+
+ |
Since print_endline expects to receive a string, the signature for hello will be:
+ |
+
+ |
But it’s not just for arguments, it’s also used when returning a value.
+
+ |
+
+ |
This function will not compile because we are trying to return a string as the first value and later an integer. +I also want to provide a larger example of the Hindley-Milner type system:
+
+ |
+
+ |
The signature for this piece of code will be:
+
+ |
+
+ |
In this example, we create a new module where we expose 3 functions: make, print_car_age, and print_car_name. We also define a type called car. One thing to note in the code is that the type is only defined once, as OCaml infers the type within the functions since car is a type within this scope.
OCaml playground for this code +Something important to note before concluding this section is that you can define both the argument types and return types for your function.
+
+ |
+
+ |
The next topic is pattern matching. I really enjoy pattern matching in programming languages. I have written a lot of Rust, and pattern matching is something I use when I write Rust. Rich pattern matching is beneficial as it eliminates the need for many if statements. Additionally, in OCaml, you are required to handle every case of the match statement.
+For example, in the code below:
+
+ |
+
+ |
In the code above, I am required to include the last match case because we have not handled every case. For example, what should the compiler do if the name is Adam? The example above is very simple. We can also match on an integer and perform different actions based on the number value. For instance, we can determine if someone is allowed to enter the party using pattern matching.
+ |
+
+ |
But the reason I mention variants in this section is that variants and pattern matching go quite nicely hand in hand. A variant is like an enumeration with more features, and I will show you what I mean. We can use them as a basic enumeration, which could look like this:
+
+ |
+
+ |
This now means that we can do different things depending on this type:
+
+ |
+
+ |
But I did mention that variants are similar to enumeration with additional features, allowing for the assignment of a type to the variant.
+
+ |
+
+ |
Now that we have added types to our variants and included HavePets, we are able to adjust our pattern matching as follows:
+ |
+
+ |
We can now assign a value to the variant and use it in pattern matching to print different values. As you can see, I am not forced to add a value to every variant. For instance, I do not need a type on HavePets so I simply don’t add it.
+I often use variants, such as in DBCaml where I use variants to retrieve responses from a database. For example, I return NoRows if I did not receive any rows back, but no error.
OCaml also comes with Exhaustiveness Checking, meaning that if we don’t check each case in a pattern matching, we will get an error. For instance, if we forget to add HavePets to the pattern matching, OCaml will throw an error at compile time.
+ |
+
+ |
The next topic is operators and specific binding operators. OCaml has more types of operators, but binding operators are something I use in every project.
+A binding could be described as something that extends how let works in OCaml by adding extra logic before storing the value in memory with let.
+I’ll show you:
+ |
+
+ |
This code simply takes the value “Emil” and stores it in memory, then assigns the memory reference to the variable hello. However, we can extend this functionality with a binding operator. For instance, if we don’t want to use a lot of match statements on the return value of a function, we can bind let so it checks the value and if the value is an error, it bubbles up the error.
+ |
+
+ |
This allows me to reduce the amount of code I write while maintaining the same functionality.
+In the code above, one of the variables is an Error, which means that the binding will return the error instead of returning the first name and last name.
I really like the concept of functional programming, such as immutability and avoiding side-effects as much as possible. However, I believe that a purely functional programming language could force us to write code in a way that becomes too complex. This is where I think OCaml does a good job. OCaml is clearly designed to be a functional language, but it allows for updating existing values rather than always returning new values.
+++Immutability means that you cannot change an already existing value and must create a new value instead. I have written about the Concepts of Functional Programming and recommend reading it if you want to learn more.
+
One example where functional programming might make the code more complex is when creating a reader to read some bytes. If we strictly follow the rule of immutability, we would need to return new bytes instead of updating existing ones. This could lead to inefficiencies in terms of memory usage.
+Just to give an example of how to mutate an existing value in OCaml, I have created an example. In the code below, I am updating the age by 1 as it is the user’s birthday:
+
+ |
+
+ |
What I mean by “it’s functional on easy mode” is simply that the language is designed to be a functional language, but you are not forced to strictly adhere to functional programming rules.
+It is clear to me that a good type system can greatly improve the developer experience. I particularly appreciate OCaml’s type system, as well as its option and result types, which I use frequently. In languages like Haskell, you can extend the type system significantly, to the point where you can write an entire application using only types. However, I believe that this can lead to overly complex code. This is another aspect of OCaml that I appreciate - it has a strong type system, but there are limitations on how far you can extend it.
I hope you enjoyed this article. If you are interested in joining a community of people who also enjoy functional programming, I recommend joining this Discord server.
+ diff --git a/data/planet/janestreet/visualizing-piecewise-linear-neural-networks.md b/data/planet/janestreet/visualizing-piecewise-linear-neural-networks.md new file mode 100644 index 0000000000..acc96718c5 --- /dev/null +++ b/data/planet/janestreet/visualizing-piecewise-linear-neural-networks.md @@ -0,0 +1,15 @@ +--- +title: Visualizing piecewise linear neural networks +description: Neural networks are often thought of as opaque, black-box function approximators, + but theoretical tools let us describe and visualize their behavior. In part... +url: https://blog.janestreet.com/visualizing-piecewise-linear-neural-networks/ +date: 2024-07-22T00:00:00-00:00 +preview_image: https://blog.janestreet.com/visualizing-piecewise-linear-neural-networks/./6_1.png +authors: +- Jane Street Tech Blog +source: +--- + +Neural networks are often thought of as opaque, black-box function approximators, but theoretical tools let us describe and visualize their behavior. In particular, let’s study piecewise-linearity, a property many neural networks share. This property has been studied before, but we’ll try to visualize it in more detail than has been previously done.
+ + diff --git a/data/planet/ocamlpro/opam-220-release.md b/data/planet/ocamlpro/opam-220-release.md new file mode 100644 index 0000000000..deed6db367 --- /dev/null +++ b/data/planet/ocamlpro/opam-220-release.md @@ -0,0 +1,246 @@ +--- +title: opam 2.2.0 release! +description: 'Feedback on this post is welcomed on Discuss! We are very pleased to + announce the release of opam 2.2.0, and encourage all users to upgrade. Please read + on for installation and upgrade instructions. NOTE: this article is cross-posted + on opam.ocaml.org and ocamlpro.com, and published in discuss.ocaml...' +url: https://ocamlpro.com/blog/2024_07_01_opam_2_2_0_releases +date: 2024-07-01T13:31:53-00:00 +preview_image: https://ocamlpro.com/assets/img/og_image_ocp_the_art_of_prog.png +authors: +- "\n Raja Boujbel - OCamlPro\n " +source: +--- + +Feedback on this post is welcomed on Discuss!
+We are very pleased to announce the release of opam 2.2.0, and encourage all users to upgrade. Please read on for installation and upgrade instructions.
+++NOTE: this article is cross-posted on opam.ocaml.org and ocamlpro.com, and published in discuss.ocaml.org.
+
In case you plan a possible rollback, you may want to first backup your
+~/.opam or $env:LOCALAPPDATAopam directory.
The upgrade instructions are unchanged:
+For Unix systems
+bash -c "sh <(curl -fsSL https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh) --version 2.2.0"
+or from PowerShell for Windows systems
+Invoke-Expression "& { $(Invoke-RestMethod https://raw.githubusercontent.com/ocaml/opam/master/shell/install.ps1) }"
+or download manually from the Github "Releases" page to your PATH.
+You should then run:
+opam init --reinit -ni
+After 8 years' effort, opam and opam-repository now have official native Windows +support! A big thank you is due to Andreas Hauptmann (@fdopen), +whose WODI and OCaml for Windows +projects were for many years the principal downstream way to obtain OCaml on +Windows, Jun Furuse (@camlspotter) whose +initial experimentation with OPAM from Cygwin +formed the basis of opam-repository-mingw, and, most recently, +Jonah Beckford (@jonahbeckford) whose +DkML distribution kept - and keeps - a full +development experience for OCaml available on Windows.
+OCaml when used on native Windows requires certain tools from the Unix world
+which are provided by either Cygwin or MSYS2.
+We have engineered opam init so that it is possible for a user not to need to
+worry about this, with opam managing the Unix world, and the user being able
+to use OCaml from either the Command Prompt or PowerShell. However, for the Unix
+user coming over to Windows to test their software, it is also possible to have
+your own Cygwin/MSYS2 installation and use native Windows opam from that. Please
+see the previous blog post
+for more information.
There are two "ports" of OCaml on native Windows, referred to by the name of +provider of the C compiler. The mingw-w64 port is GCC-based. +opam's external dependency (depext) system works for this port (including +providing GCC itself), and many packages are already well-supported in +opam-repository, thanks to the previous efforts in opam-repository-mingw. +The MSVC port is Visual Studio-based. At +present, there is less support in this ecosystem for external dependencies, +though this is something we expect to work on both in opam-repository and in +subsequent opam releases. In particular, it is necessary to install +Visual Studio or Visual Studio BuildTools separately, but opam will then +automatically find and use the C compiler from Visual Studio.
+opam tree is a new command showing packages and their dependencies with a tree view.
+It is very helpful to determine which packages bring which dependencies in your installed switch.
$ opam tree cppo
+cppo.1.6.9
+├── base-unix.base
+├── dune.3.8.2 (>= 1.10)
+│ ├── base-threads.base
+│ ├── base-unix.base [*]
+│ └── ocaml.4.14.1 (>= 4.08)
+│ ├── ocaml-base-compiler.4.14.1 (>= 4.14.1~ & < 4.14.2~)
+│ └── ocaml-config.2 (>= 2)
+│ └── ocaml-base-compiler.4.14.1 (>= 4.12.0~) [*]
+└── ocaml.4.14.1 (>= 4.02.3) [*]
+Reverse-dependencies can also be displayed using the new opam why command.
+This is useful to examine how dependency versions get constrained.
$ opam why cmdliner
+cmdliner.1.2.0
+├── (>= 1.1.0) b0.0.0.5
+│ └── (= 0.0.5) odig.0.0.9
+├── (>= 1.1.0) ocp-browser.1.3.4
+├── (>= 1.0.0) ocp-indent.1.8.1
+│ └── (>= 1.4.2) ocp-index.1.3.4
+│ └── (= version) ocp-browser.1.3.4 [*]
+├── (>= 1.1.0) ocp-index.1.3.4 [*]
+├── (>= 1.1.0) odig.0.0.9 [*]
+├── (>= 1.0.0) odoc.2.2.0
+│ └── (>= 2.0.0) odig.0.0.9 [*]
+├── (>= 1.1.0) opam-client.2.2.0~alpha
+│ ├── (= version) opam.2.2.0~alpha
+│ └── (= version) opam-devel.2.2.0~alpha
+├── (>= 1.1.0) opam-devel.2.2.0~alpha [*]
+├── (>= 0.9.8) opam-installer.2.2.0~alpha
+└── user-setup.0.7
+++Special thanks to @cannorin for contributing this feature.
+
There is now a way for a project maintainer to share their project development
+tools: the with-dev-setup dependency flag. It is used in the same way as
+with-doc and with-test: by adding a {with-dev-setup} filter after a
+dependency. It will be ignored when installing normally, but it's pulled in when the
+package is explicitly installed with the --with-dev-setup flag specified on
+the command line.
For example
+opam-version: "2.0"
+depends: [
+ "ocaml"
+ "ocp-indent" {with-dev-setup}
+]
+build: [make]
+install: [make "install"]
+post-messages:
+[ "Thanks for installing the package"
+ "as well as its development setup. It will help with your future contributions" {with-dev-setup} ]
+When pinning a package using opam pin, opam looks for opam files in the root directory only.
+With recursive pinning, you can now instruct opam to look for .opam files in
+subdirectories as well, while maintaining the correct relationship between the .opam
+files and the package root for versioning and build purposes.
Recursive pinning is enabled by the following options to opam pin and opam install:
--recursive, opam will look for .opam files recursively in all subdirectories.
+--subpath <path>, opam will only look for .opam files in the subdirectory <path>.
+The two options can be combined: for instance, if your opam packages are stored
+as a deep hierarchy in the mylib subdirectory of your project you can try
+opam pin . --recursive --subpath mylib.
These options are useful when dealing with a large monorepo-type repository with many +opam libraries spread about.
+opam switch -, inspired by git switch -, makes opam switch back to the previously
+selected global switch.
opam pin --current fixes a package to its current state (disabling pending
+reinstallations or removals from the repository). The installed package will
+be pinned to its current installed state, i.e. the pinned opam file is the
+one installed.
opam pin remove --all removes all the pinned packages from a switch.
opam exec --no-switch removes the opam environment when running a command.
+It is useful when you want to launch a command without opam environment changes.
opam clean --untracked removes untracked files interactively remaining
+from previous packages removal.
opam admin add-constraint <cst> --packages pkg1,pkg2,pkg3 applies the given constraint
+to a given set of packages
opam list --base has been renamed into --invariant, reflecting the fact that since opam 2.1 the "base" packages of a switch are instead expressed using a switch invariant.
opam install --formula <formula> installs a formula instead of a list of packages. This can be useful if you would like to install one package or another one. For example opam install --formula '"extlib" |"extlib-compat"' will install either extlib or extlib-compat depending on what's best for the current switch.
opam env, fixing many corner cases for environment updates and making the reverting of package environment variables precise. As a result, using setenv in an opam file no longer triggers a lint warning.
+sys-ocaml-system default global eval variable
+"%{var?string-if-true:string-if-false-or-undefined}%" syntax to
+support extending the variables of packages with + in their name
+(conf-c++ and conf-g++ already exist) using "%{?pgkname:var:}%"
+getconf DARWIN_USER_TEMP_DIR) as writable when TMPDIR
+is not defined on macOS
+/tmp is now writable again, restoring POSIX compliance
+opam admin: new add-extrafiles command to add/check/update the extra-files: field according to the files present in the files/ directory
+opam lint -W @1..9 syntax to allow marking a set of warnings as errors
+OPAMCURL, OPAMFETCH and OPAMVERBOSE environment variables
+--assume-built argument
+And many other general and performance improvements were made and bugs were fixed. +You can take a look to previous blog posts. +API changes and a more detailed description of the changes are listed in:
+This release also includes PRs improving the documentation and improving +and extending the tests.
+Please report any issues to the bug-tracker.
+We hope you will enjoy the new features of opam 2.2! 📯
+ diff --git a/data/planet/signalsandthreads/from-the-lab-to-the-trading-floor-with-erin-murphy.md b/data/planet/signalsandthreads/from-the-lab-to-the-trading-floor-with-erin-murphy.md new file mode 100644 index 0000000000..d86765879d --- /dev/null +++ b/data/planet/signalsandthreads/from-the-lab-to-the-trading-floor-with-erin-murphy.md @@ -0,0 +1,13 @@ +--- +title: From the Lab to the Trading Floor with Erin Murphy +description: +url: https://signals-threads.simplecast.com/episodes/from-the-lab-to-the-trading-floor-with-erin-murphy-hD6GHMhc +date: 2024-07-12T19:15:09-00:00 +preview_image: +authors: +- Signals and Threads +source: +--- + +Erin Murphy is Jane Street’s first UX designer, and before that, she worked at NASA’s Jet Propulsion Laboratory building user interfaces for space missions. She’s also an illustrator with her own quarterly journal. In this episode, Erin and Ron discuss the challenge of doing user-centered design in an organization where experts are used to building tools for themselves. How do you bring a command-line interface to the web without making it worse for power users? They also discuss how beauty in design is more about utility than aesthetics; what Jane Street looks for in UX candidates; and how to help engineers discover what their users really want.
You can find the transcript for this episode on our website.
Some links to topics that came up in the discussion:
The last post looked at using various tools to understand why an OCaml 5 program was waiting a long time for IO. +In this post, I'll be trying out some tools to investigate a compute-intensive program that uses multiple CPUs.
+ +Table of Contents
+Further discussion about this post can be found on discuss.ocaml.org.
+OCaml 4 allowed running multiple "system threads", but only one can have the OCaml runtime lock, +so only one can be running OCaml code at a time. +OCaml 5 allows running multiple "domains", all of which can be running OCaml code at the same time +(each domain can also have multiple system threads; only one system thread can be running OCaml code per domain).
+The ocaml-ci service provides CI for many OCaml programs, +and its first step when testing a commit is to run a solver to select compatible versions for its dependencies. +Running a solve typically only takes about a second, but it has to do it for each possible test platform, +which includes versions of the OCaml compiler from 4.02 to 4.14 and 5.0 to 5.2, +multiple architectures (32-bit and 64-bit x86, 32-bit and 64-bit ARM, PPC64 and s390x), +operating systems (Alpine, Debian, Fedora, FreeBSD, macos, OpenSUSE and Ubuntu, in multiple versions), etc. +In total, this currently does 132 solver runs per commit being tested +(which seems too high to me, but let's ignore that for now).
+The solves are done by the solver-service, +which runs on a couple of ARM machines with 160 cores each. +The old OCaml 4 version used to work by spawning lots of sub-processes, +but when OCaml 5 came out, I ported it to use a single process with multiple domains. +That removed the need for lots of communication logic, +and allowed sharing common data such as the package definitions. +The code got a lot shorter and simpler, and I'm told it's been much more reliable too.
+But the performance was surprisingly bad. +Here's a graph showing how the number of solves per second scales with the number of CPUs (workers) being used:
+Processes scaling better than domains
The "Processes" line shows performance when forking multiple processes to do the work, which looks pretty good. +The "Domains" line shows what happens if you instead spawn domains inside a single process.
+Note: The original service used many libraries (a mix of Eio and Lwt ones), +but to make investigation easier I simplified it by removing most of them. +The simplified version doesn't use Eio or Lwt; +it just spawns some domains/processes and has each of them do the same solve in a loop a fixed number of times.
+When converting a single-domain OCaml 4 program to use multiple cores it's easy to introduce races.
+OCaml has ThreadSanitizer (TSan) support which can detect these.
+To use it, install an OCaml compiler with the tsan option:
$ opam switch create 5.2.0-tsan ocaml-variants.5.2.0+options ocaml-option-tsan
+Things run a lot slower and require more memory with this compiler, but it's good to check:
+$ ./_build/default/stress/stress.exe --internal-workers=2
+[...]
+WARNING: ThreadSanitizer: data race (pid=133127)
+ Write of size 8 at 0x7ff2b7814d38 by thread T4 (mutexes: write M88):
+ #0 camlOpam_0install__Model.group_ors_1288 lib/model.ml:70 (stress.exe+0x1d2bba)
+ #1 camlOpam_0install__Model.group_ors_1288 lib/model.ml:120 (stress.exe+0x1d2b47)
+ ...
+
+ Previous write of size 8 at 0x7ff2b7814d38 by thread T1 (mutexes: write M83):
+ #0 camlOpam_0install__Model.group_ors_1288 lib/model.ml:70 (stress.exe+0x1d2bba)
+ #1 camlOpam_0install__Model.group_ors_1288 lib/model.ml:120 (stress.exe+0x1d2b47)
+ ...
+
+ Mutex M88 (0x558368b95358) created at:
+ #0 pthread_mutex_init ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1295 (libtsan.so.2+0x50468)
+ #1 caml_plat_mutex_init runtime/platform.c:57 (stress.exe+0x4763b2)
+ #2 caml_init_domains runtime/domain.c:943 (stress.exe+0x44ebfe)
+ ...
+
+ Mutex M83 (0x558368b95240) created at:
+ #0 pthread_mutex_init ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1295 (libtsan.so.2+0x50468)
+ #1 caml_plat_mutex_init runtime/platform.c:57 (stress.exe+0x4763b2)
+ #2 caml_init_domains runtime/domain.c:943 (stress.exe+0x44ebfe)
+ ...
+
+ Thread T4 (tid=133132, running) created by main thread at:
+ #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1001 (libtsan.so.2+0x5e686)
+ #1 caml_domain_spawn runtime/domain.c:1265 (stress.exe+0x4504c4)
+ ...
+
+ Thread T1 (tid=133129, running) created by main thread at:
+ #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1001 (libtsan.so.2+0x5e686)
+ #1 caml_domain_spawn runtime/domain.c:1265 (stress.exe+0x4504c4)
+ ...
+
+SUMMARY: ThreadSanitizer: data race lib/model.ml:70 in camlOpam_0install__Model.group_ors_1288
+The two mutexes mentioned in the output, M83 and M88, are the domain_lock,
+used to ensure only one sys-thread runs at a time in each domain.
+In this program we only have one sys-thread per domain and so can ignore them.
The output reveals that the solver used a global variable to generate unique IDs:
+1 +2 +3 +4 +5 + | |
With that fixed, TSan finds no further problems (in this simplified version). +This gives us good confidence that there isn't any shared state: +TSan would report use of shared state not protected by a mutex, +and since the program was written for OCaml 4 it won't be using any mutexes.
+That's good, because if one thread writes to a location that another reads then that requires coordination between CPUs, +which is relatively slow +(though we could still experience slow-downs due to false sharing, +where two separate mutable items end up in the same cache line). +However, while important for correctness, it didn't make any noticeable difference to the benchmark results.
+perf is the obvious tool to use when facing CPU performance problems.
+perf record -g PROG takes samples of the program's stack regularly,
+so that functions that run a lot or for a long time will appear often.
+perf report provides a UI to explore the results:
$ perf report
+ Children Self Command Shared Object Symbol
++ 59.81% 0.00% stress.exe stress.exe [.] Zeroinstall_solver.Solver_core.do_solve_2283
++ 59.44% 0.00% stress.exe stress.exe [.] Opam_0install.Solver.solve_1428
++ 59.25% 0.00% stress.exe stress.exe [.] Dune.exe.Domain_worker.solve_951
++ 58.88% 0.00% stress.exe stress.exe [.] Dune.exe.Stress.run_worker_332
++ 58.18% 0.00% stress.exe stress.exe [.] Stdlib.Domain.body_735
++ 57.91% 0.00% stress.exe stress.exe [.] caml_start_program
++ 34.39% 0.69% stress.exe stress.exe [.] Stdlib.List.iter_366
++ 34.39% 0.03% stress.exe stress.exe [.] Zeroinstall_solver.Solver_core.lookup_845
++ 34.39% 0.09% stress.exe stress.exe [.] Zeroinstall_solver.Solver_core.process_dep_2024
++ 33.14% 0.03% stress.exe stress.exe [.] Zeroinstall_solver.Sat.run_solver_1446
++ 27.28% 0.00% stress.exe stress.exe [.] Zeroinstall_solver.Solver_core.build_problem_2092
++ 26.27% 0.02% stress.exe stress.exe [.] caml_call_gc
+Looks like we're spending most of our time solving, as expected. +But this can be misleading. +Because perf only records stack traces when the code is running, it doesn't report any time the process spent sleeping.
+$ /usr/bin/time ./_build/default/stress/stress.exe --count=10 --internal-workers=7
+73.08user 0.61system 0:12.65elapsed 582%CPU (0avgtext+0avgdata 596608maxresident)k
+With 7 workers, we'd expect to see 700%CPU, but we only see 582%.
mpstat can show a per-CPU breakdown. +Here are a couple of one second intervals on my machine while the solver was running:
+$ mpstat --dec=0 -P ALL 1
+16:24:39 CPU %usr %sys %iowait %irq %soft %steal %idle
+16:24:40 all 78 1 2 1 0 0 18
+16:24:40 0 19 1 0 1 0 1 78
+16:24:40 1 88 1 0 1 0 0 10
+16:24:40 2 88 1 0 1 0 0 10
+16:24:40 3 88 0 0 0 0 1 11
+16:24:40 4 89 1 0 0 0 0 10
+16:24:40 5 90 0 0 1 0 0 9
+16:24:40 6 79 1 0 1 1 1 17
+16:24:40 7 86 0 12 1 1 0 0
+
+16:24:40 CPU %usr %sys %iowait %irq %soft %steal %idle
+16:24:41 all 80 1 2 1 0 0 17
+16:24:41 0 85 0 12 1 0 1 1
+16:24:41 1 91 1 0 1 0 0 7
+16:24:41 2 90 0 0 1 1 0 8
+16:24:41 3 89 1 0 1 0 0 9
+16:24:41 4 67 1 0 1 0 0 31
+16:24:41 5 52 1 0 0 0 1 46
+16:24:41 6 76 1 0 1 0 0 22
+16:24:41 7 90 1 0 0 0 0 9
+Note: I removed some columns with all zero values to save space.
+We might expect to see 7 CPUs running at 100% and one idle CPU, +but in fact they're all moderately busy. +On the other hand, none of them spent more than 91% of its time running the solver code.
+offcputime will show why a process wasn't using a CPU
+(it's like offwaketime, which we saw earlier, but doesn't record the waker).
+Here I'm using pidstat to see all running threads and then examining one of the workers,
+to avoid the problem we saw last time where the diagram included multiple threads:
$ pidstat 1 -t
+...
+^C
+Average: UID TGID TID %usr %system %guest %wait %CPU CPU Command
+Average: 1000 78304 - 550.50 9.41 0.00 0.00 559.90 - stress.exe
+Average: 1000 - 78305 91.09 1.49 0.00 0.00 92.57 - |__stress.exe
+Average: 1000 - 78307 8.42 0.99 0.00 0.00 9.41 - |__stress.exe
+Average: 1000 - 78308 90.59 1.49 0.00 0.00 92.08 - |__stress.exe
+Average: 1000 - 78310 90.59 1.49 0.00 0.00 92.08 - |__stress.exe
+Average: 1000 - 78312 91.09 1.49 0.00 0.00 92.57 - |__stress.exe
+Average: 1000 - 78314 89.11 1.49 0.00 0.00 90.59 - |__stress.exe
+Average: 1000 - 78316 89.60 1.98 0.00 0.00 91.58 - |__stress.exe
+
+$ sudo offcputime-bpfcc -f -t 78310 > off-cpu
+Note: The ARM machine's kernel was too old to run offcputime, so I ran this on my machine instead,
+with one main domain and six workers.
+As I needed good stacks for C functions too, I ran stress.exe in an Ubuntu 24.04 docker container,
+as recent versions of Ubuntu compile with frame pointers by default.
The raw output was very noisy, showing it waiting in many different places.
+Looking at a few, it was clear it was mostly the GC (which can run from almost anywhere).
+The output is just a text-file with one line per stack-trace, and bit of sed cleaned it up:
$ sed -E 's/stress.exe;.*;(caml_call_gc|caml_handle_gc_interrupt|caml_poll_gc_work|asm_sysvec_apic_timer_interrupt|asm_sysvec_reschedule_ipi);/stress.exe;\\1;/' off-cpu > off-cpu-gc
+$ flamegraph.pl --colors=blue off-cpu-gc > off-cpu-gc.svg
+That removes the part of the stack-trace before any of various interrupt-type functions that can be called from anywhere. +The graph is blue to indicate that it shows time when the process wasn't running.
+ +There are rather a lot of traces where we missed the user stack.
+However, the results seem clear enough: when our worker is waiting, it's in the garbage collector,
+calling caml_plat_spin_wait.
+This is used to sleep when a spin-lock has been spinning for too long (after 1000 iterations).
OCaml has a major heap for long-lived values, plus one fixed-size minor heap for each domain. +New allocations are made sequentially on the allocating domain's minor heap +(which is very fast, just adjusting a pointer by the size required).
+When the minor heap is full the program performs a minor GC, +moving any values that are still reachable to the major heap +and leaving the minor heap empty.
+Garbage collection of the major heap is done in small slices so that the application doesn't pause for long, +and domains can do marking and sweeping work without needing to coordinate +(except at the very end of a major cycle, when they briefly synchronise to agree a new cycle is starting).
+However, as minor GCs move values that other domains may be using, they do require all domains to stop.
+Although the simplified test program doesn't use Eio, we can still use eio-trace to record GC events +(we just don't see any fibers). +Here's a screenshot of the solver running with 24 domains on the ARM machine, +showing it performing GC work (not all domains are visible in the picture):
+ + +The orange/red parts show when the GC is running and the yellow regions show when the domain is waiting for other domains. +The thick columns with yellow edges are minor GCs, +while the thin (almost invisible) red columns without any yellow between them are major slices. +The second minor GC from the left took longer than usual because the third domain from the top took a while to respond. +It also didn't do a major slice before that; perhaps it was busy doing something, or maybe Linux scheduled a different process to run then.
+Traces recorded by eio-trace can also be viewed in Perfetto, which shows the nesting better: +Here's a close-up of a single minor GC, corresponding to the bottom two domains from the second column from the left:
+ +We can now see why the solver spends so much time sleeping; +when a domain performs a minor GC, it spends most of the time waiting for other domains.
+(the above is a slight simplification; domains may do some work on the major GC while waiting)
+One obvious solution to GC slowness is to produce less garbage in the first place. +To do that, we need to find out where the most costly allocations are coming from. +Tracing every memory allocation tends to make programs unusably slow, +so OCaml instead provides a statistical memory profiler.
+It was temporarily removed in OCaml 5 because it needed updating for the new multicore GC, +but has recently been brought back and will be in OCaml 5.3. +There's a backport to 5.2, but I couldn't get it to work, +so I just removed the domains stuff from the test and did a single-domain run on OCaml 4.14. +You need the memtrace library to collect samples and memtrace_viewer to view them:
+
$ opam install memtrace memtrace_viewer
+Put this at the start of the program to enable it:
+1
+ | |
Then running with MEMTRACE set records a trace:
$ MEMTRACE=solver.ctf ./stress.exe --count=10
+Solved warm-up request in: 1.99s
+Running another 10 * 1 solves...
+
+$ memtrace-viewer solver.ctf
+Processing solver.ctf...
+Serving http://localhost:8080/
+The flame graph in the middle shows functions scaled by the amount of memory they allocated.
+Initially it showed two groups, one for the warm-up request and one for the 10 runs.
+To simplify the display, I used the filter panel (on the left) to show only allocations after the 2 second warm-up.
+We can immediately see that OpamVersionCompare.compare is the source of most memory use.
Focusing on that function shows that it performed 54.1% of all allocations. +The display now shows allocations performed within it above it (in green), +and all the places it's called from in blue below:
+
The compare function is expensive!
The bulk of the allocations are coming from this loop:
+1 +2 +3 +4 +5 +6 +7 +8 +9 +10 + | |
It's used when processing a version like 1.2.3 to skip any leading "0" characters
+(so that would compare equal to 1.02.3).
+The loop function refers to other variables (such as f) from its context,
+and so OCaml allocates a closure on the heap to hold these variables.
+Even though these allocations are small, we have to do it for every component of every version.
+And we compare versions a lot:
+for every version of a package that says it requires e.g. libfoo { >= "1.2" },
+we have to check the formula against every version of libfoo.
The solution is rather simple (and shorter than the original!):
+1 +2 +3 + | |
Removing the other allocations from compare too reduces total memory allocations
+from 21.8G to 9.6G!
+The processes benchmark got about 14% faster, while the domains one was 23% faster:
Effect of reducing allocations. Old values are shown in grey.
A nice optimisation, +but using domains is still nowhere close to even the original version with separate processes.
+The traces above show the solver taking a long time for all domains to enter the stw_api_barrier phase.
+What was the slow domain doing to cause that?
+magic-trace let's us tell it when to save the ring buffer and we can use this to get detailed information.
+Tracing multiple threads with magic-trace doesn't seem to work well
+(each thread gets a very small buffer, they don't stop at quite the same time, and triggers don't work)
+so I find it's better to trace just one thread.
I modified the OCaml runtime so that the leader (the domain requesting the GC) records the time.
+As each domain enters stw_api_barrier it checks how late it is and calls a function to print a warning if it's above a threshold.
+Then I attached magic-trace to one of the worker threads and told it to save a sample when that function got called:
A domain being slow to join a minor GC
In the example above,
+magic-trace saved about 7ms of the history of a domain up to the point where it entered stw_api_barrier.
+The first few ms show the solver working normally.
+Then it needs to do a minor GC and tries to become the leader.
+But another domain has the lock and so it spins, calling handle_incoming 293,711 times in a loop for 2.5ms.
I had a look at the code in the OCaml runtime. +When a domain wants to perform a minor GC, the steps are:
+
all_domains_lock.
+stw_request global.
+all_domains_lock.
+all_domains_cond and everyone resumes.
+I added some extra event reporting to the GC, showing when a domain is trying to perform a GC (try),
+when the leader is signalling other domains (signal), and when a domain is sleeping waiting for something (sleep).
+Here's what that looks like (in some places):
One sleeping domain delays all the others
These try events seem useful;
+the program is spending much more time stuck in GC than the original traces indicated!
One obvious improvement here would be for idle domains to opt out of GC. +Another would be to tell the kernel when to wake instead of using sleeps — +and I see there's a PR already: +OS-based Synchronisation for Stop-the-World Sections.
+Another possibility would be to let domains perform minor GCs independently. +The OCaml developers did make a version that worked that way, +but it requires changes to all C code that uses the OCaml APIs, +since a value in another domain's minor heap might move while it's running.
+Finally, I wonder if the code could be simplified a bit using a compare-and-set instead of taking a lock to become leader.
+That would eliminate the try state, where a domain knows another domain is the leader, but doesn't know what it wants to do.
+It's also strange that there's a state where
+the top domain has finished its critical section and allowed the other domains to resume,
+but is not quite finished enough to let a new GC start.
We can work around this problem by having the main domain do work too. +That could be a problem for interactive applications (where the main domain is running the UI and needs to respond fast), +but it should be OK for the solver service. +This was about 15% faster on my machine, but appeared to have no effect on the ARM server. +Lesson: get traces on the target machine!
+Another way to reduce the synchronisation overhead of minor GCs is to make them less frequent.
+We can do that by increasing the size of the minor heap,
+doing a few long GCs rather than many short ones.
+The size is controlled by the setting e.g. OCAMLRUNPARAM=s=8192k.
+On my machine, this actually makes things slower, but it's about 18% faster on the ARM server with 80 domains.
Here are the first few domains (from a total of 24) on the ARM server with different minor heap sizes +(both are showing 1s of execution):
+
The default minor heap size (256k words)
+
With a larger minor heap (8192k works)
+Note that the major slices also get fewer and larger, as they happen half way between minor slices.
Also, there's still a lot of variation between the time each domain spends doing GC +(despite the fact that they're all running exactly the same task), so they still end up waiting a lot.
+This is all still pretty odd, though. +We're getting small performance increases, but still nothing like when forking. +Can the test-case be simplified further? +Yes, it turns out! +This simple function takes much longer to run when using domains, compared to forking!
+1 +2 +3 +4 + | |
ref () allocates a small block (2 words, including the header) on the minor heap.
+opaque_identity is to make sure the compiler doesn't optimise this pointless allocation away.
Time to run the loop on the 160-core ARM server (lower is better)
Here's what I would expect here:
+So ideally the lines would be flat. +In practice, we may hit physical limits due to memory bandwidth, CPU temperature or kernel limitations; +I assume this is why the "Processes" time starts to rise eventually. +But it looks like this minor slow-down causes knock-on effects in the "Domains" case.
+If I remove the allocation, then the domains and processes versions take the same amount of time.
+perf sched record records kernel scheduling events, allowing it to show what is running on each CPU at all times.
+perf sched timehist displays a report:
$ sudo perf sched record -k CLOCK_MONOTONIC
+^C
+
+$ sudo perf sched timehist
+ time cpu task name wait time sch delay run time
+ [tid/pid] (msec) (msec) (msec)
+--------------- ------ ------------------------------ --------- --------- ---------
+ 185296.715345 [0000] sway[175042] 1.694 0.025 0.775
+ 185296.716024 [0002] crosvm_vcpu2[178276/178217] 0.012 0.000 2.957
+ 185296.717031 [0003] main.exe[196519] 0.006 0.000 4.004
+ 185296.717044 [0003] rcu_preempt[18] 4.004 0.015 0.012
+ 185296.717260 [0001] main.exe[196526] 1.760 0.000 2.633
+ 185296.717455 [0001] crosvm_vcpu1[193502/193445] 63.809 0.015 0.194
+ ...
+The first line here shows that sway needed to wait for 1.694 ms for some reason (possibly a sleep),
+and then once it was due to resume, had to wait a further 0.025 ms for CPU 0 to be free. It then ran for 0.775 ms.
+I decided to use perf sched to find out what the system was doing when a domain failed to respond quickly.
To make the output easier to read, I hacked eio-trace to display it on the traces.
+perf script -g python will generate a skeleton Python script that can format all the events found in the perf.data file,
+and I used that to convert the output to CSV.
+To correlate OCaml domains with Linux threads, I also modified OCaml to report the thread ID (TID) for each new domain
+(it was previously reporting the PID instead for some reason).
Here's a trace of the simple allocator from the previous section:
+
eio-trace with perf sched data
Note: the colour of stw_api_barrier has changed: previously eio-trace coloured it yellow to indicate sleeping,
+but now we have the individual sleep events we can see exactly which part of it was sleeping.
The horizontal green bars show when each domain was running on the CPU.
+Here, we see that most of the domains ran until they called sleep.
+When the sleep timeout expires, the thread is ready to run again and goes on the run-queue.
+Time spent waiting on the queue is shown with a black bar.
When switching to or from another process, the process name is shown.
+Here we can see that crosvm_vcpu6 interrupted one of our domains, making it late to respond to the GC request.
Here we see another odd feature of the protocol: even though the late domain was the last to be ready, +it wasn't able to start its GC even then, because only the leader is allowed to say when everyone is ready. +Several domains wake after the late one is ready and have to go back to sleep again.
+The diagram also shows when Linux migrated our OCaml domains between CPUs. +For example:
+Here's another example:
+ +I tried using the processor package to pin each domain to a different CPU. +That cleaned up the traces a fair bit, but didn't make much difference to the runtime on my machine.
+I also tried using chrt to run the program as a high-priority "real-time" task,
+which also didn't seem to help.
+I wrote a bpftrace script to report if one of our domains was ready to resume and the scheduler instead ran something else.
+That showed various things.
+Often Linux was migrating something else out of the way and we had to wait for that,
+but there were also some kernel tasks that seemed to be even higher priority, such as GPU drivers or uring workers.
+I suspect to make this work you'd need to set the affinity of all the other processes to keep them away from the cores being used
+(but that wouldn't work in this example because I'm using all of them!).
+Come to think of it, running a CPU intensive task on every CPU at realtime priority was a dumb idea;
+had it worked I wouldn't have been able to do anything else with the computer!
Exploring the scheduler behaviour was interesting, and might be needed for latency-sensitive tasks, +but how often do migrations and delays really cause trouble? +The slow GCs are interesting, but there are also sections like this where everything is going smoothly, +and minor GCs take less than 4 microseconds:
+ +olly can be used get summary statistics:
+
$ olly gc-stats './_build/default/stress/stress.exe --count=6 --internal-workers=24'
+...
+Solved 144 requests in 25.44s (0.18s/iter) (5.66 solves/s)
+
+Execution times:
+Wall time (s): 28.17
+CPU time (s): 1.66
+GC time (s): 169.88
+GC overhead (% of CPU time): 10223.84%
+
+GC time per domain (s):
+Domain0: 0.47
+Domain1: 9.34
+Domain2: 6.90
+Domain3: 6.97
+Domain4: 6.68
+Domain5: 6.85
+Domain6: 6.59
+...
+10223.84% GC overhead sounds like a lot but I think this is a misleading, for a few reasons:
+time reports about 6 minutes which sounds more likely.
+To double-check, I modified eio-trace to report GC statistics for a saved trace:
+Solved 144 requests in 26.84s (0.19s/iter) (5.36 solves/s)
+...
+
+$ eio-trace gc-stats trace.fxt
+./trace.fxt:
+
+Ring GC/s App/s Total/s %GC
+ 0 10.255 19.376 29.631 34.61
+ 1 7.986 10.201 18.186 43.91
+ 2 8.195 10.648 18.843 43.49
+ 3 9.521 14.398 23.919 39.81
+ 4 9.775 16.537 26.311 37.15
+ 5 8.084 10.635 18.719 43.19
+ 6 7.977 10.356 18.333 43.51
+...
+ 24 7.920 10.802 18.722 42.30
+
+All 213.332 308.578 521.910 40.88
+
+Note: all times are wall-clock and so include time spent blocking.
+It ran slightly slower under eio-trace, perhaps because recording a trace file is more work than maintaining some counters, +but it's similar. +So this indicates that with 24 domains GC is taking about 40% of the total time (including time spent sleeping).
+But something doesn't add up, on my machine at least:
+Even if that 20% were removed completely, it should only save 20% of the 8.2s. +So with domains, the code must be running more slowly even when it's not in the GC.
+I tried running magic-trace to see what it was doing outside of the GC. +Since it wasn't calling any functions, it didn't show anything, but we can fix that:
+1 +2 +3 +4 +5 +6 +7 +8 +9 +10 + | |
Here we do blocks of 100 allocations in a function called foo.
+The annotations are to ensure the compiler doesn't inline it.
+The trace was surprisingly variable!
magic-trace of foo between GCs
I see times for foo ranging from 50ns to around 750ns!
Note: the extra foo call above was probably due to a missed end event somewhere.
I ran perf record on the simplified version:
1 +2 +3 +4 + | |
Here the code is simple enough that we don't need stack-traces (so no -g):
$ sudo perf record ./_build/default/main.exe
+$ sudo perf annotate
+
+ │ camlDune__exe__Main.foo_273():
+ │ mov $0x3,%eax
+ 0.04 │ cmp $0xc9,%rax
+ │ ↓ jg 39
+ 7.34 │ d: sub $0x10,%r15
+ 13.37 │ cmp (%r14),%r15
+ 0.09 │ ↓ jb 3f
+ 0.21 │16: lea 0x8(%r15),%rbx
+ 70.26 │ movq $0x400,-0x8(%rbx)
+ 6.66 │ movq $0x1,(%rbx)
+ 0.73 │ mov %rax,%rbx
+ 0.00 │ add $0x2,%rax
+ 0.01 │ cmp $0xc9,%rbx
+ 0.66 │ ↑ jne d
+ 0.28 │39: mov $0x1,%eax
+ 0.34 │ ← ret
+ 0.00 │3f: → call caml_call_gc
+ │ ↑ jmp 16
+The code starts by (pointlessly) checking if 1 > 100 in case it can skip the whole loop. +After being disappointed, it:
+%r15 (young_ptr) by 0x10 (two words).
+young_limit, calling caml_call_gc if so to clear the minor heap.
+().
+().
+Looks like we spent most of the time (77%) writing the block, which makes sense.
+Reading young_limit took 13% of the time, which seems reasonable too.
+If there was contention between domains, we'd expect to see it here.
The output looked similar whether using domains or processes.
+To double-check, I also tried perf c2c.
+This reports on cache-to-cache transfers, where two CPUs are accessing the same memory,
+which requires the processors to communicate and is therefore relatively slow.
$ sudo perf c2c record
+^C
+
+$ sudo perf c2c report
+ Load Operations : 11898
+ Load L1D hit : 4140
+ Load L2D hit : 93
+ Load LLC hit : 3750
+ Load Local HITM : 251
+ Store Operations : 116386
+ Store L1D Hit : 104763
+ Store L1D Miss : 11622
+...
+# ----- HITM ----- ------- Store Refs ------ ------- CL -------- ---------- cycles ---------- Total cpu Shared
+# RmtHitm LclHitm L1 Hit L1 Miss N/A Off Node PA cnt Code address rmt hitm lcl hitm load records cnt Symbol Object Source:Line Node
+...
+ 7 0 7 4 0 0 0x7f90b4002b80
+ ----------------------------------------------------------------------
+ 0.00% 100.00% 0.00% 0.00% 0.00% 0x0 0 1 0x44a704 0 144 107 8 1 [.] Dune.exe.Main.foo_273 main.exe main.ml:7 0
+ 0.00% 0.00% 25.00% 0.00% 0.00% 0x0 0 1 0x4ba7b9 0 0 0 1 1 [.] caml_interrupt_all_signal_ main.exe domain.c:318 0
+ 0.00% 0.00% 25.00% 0.00% 0.00% 0x0 0 1 0x4ba7e2 0 0 323 49 1 [.] caml_reset_young_limit main.exe domain.c:1658 0
+ 0.00% 0.00% 25.00% 0.00% 0.00% 0x8 0 1 0x4ce94d 0 0 0 1 1 [.] caml_empty_minor_heap_prom main.exe minor_gc.c:622 0
+ 0.00% 0.00% 25.00% 0.00% 0.00% 0x8 0 1 0x4ceed2 0 0 0 1 1 [.] caml_alloc_small_dispatch main.exe minor_gc.c:874 0
+This shows a list of cache lines (memory addresses) and how often we loaded from a modified address.
+There's a lot of information here and I don't understand most of it.
+But I think the above is saying that address 0x7f90b4002b80 (young_limit, at offsets 0) was accessed by these places across domains:
main.ml:7 (ref ()) checks against young_limit to see if we need to call into the GC.
+domain.c:318 sets the limit to UINTNAT_MAX to signal that another domain wants a GC.
+domain.c:1658 sets it back to young_trigger after being signalled.
+The same cacheline was also accessed at offset 8, which contains young_ptr (address of last allocation):
minor_gc.c:622 sets young_ptr to young_end after a GC.
+minor_gc.c:874 adjusts young_ptr to re-do the allocation that triggered the GC.
+This indicates false sharing: young_ptr only gets accessed from one domain but it's in the same cache line as young_limit.
The main thing is that the counts are all very low, indicating that this doesn't happen often.
+I tried adding an incr x on a global variable in the loop, and got some more operations reported.
+But using Atomic.incr massively increased the number of records:
| Original | incr | Atomic.incr | |
|---|---|---|---|
| Load Operations | 11,898 | 25,860 | 2,658,364 |
| Load L1D hit | 4,140 | 15,181 | 326,236 |
| Load L2D hit | 93 | 163 | 295 |
| Load LLC hit | 3,750 | 3,173 | 2,321,704 |
| Load Local HITM | 251 | 299 | 2,317,885 |
| Store Operations | 116,386 | 462,162 | 3,909,500 |
| Store L1D Hit | 104,763 | 389,492 | 3,908,947 |
| Store L1D Miss | 11,622 | 72,667 | 550 |
See C2C - False Sharing Detection in Linux Perf for more information about all this.
+perf stat shows statistics about a process.
+I ran it with -I 1000 to collect one-second samples.
+Here are two samples from the test case on my machine,
+one when it was running processes and one while it was using domains:
$ perf stat -I 1000
+
+# Processes
+ 8,032.71 msec cpu-clock # 8.033 CPUs utilized
+ 2,475 context-switches # 308.115 /sec
+ 51 cpu-migrations # 6.349 /sec
+ 44 page-faults # 5.478 /sec
+35,268,665,452 cycles # 4.391 GHz
+48,673,075,188 instructions # 1.38 insn per cycle
+ 9,815,905,270 branches # 1.222 G/sec
+ 48,986,037 branch-misses # 0.50% of all branches
+
+# Domains
+ 8,008.11 msec cpu-clock # 8.008 CPUs utilized
+ 10,970 context-switches # 1.370 K/sec
+ 133 cpu-migrations # 16.608 /sec
+ 232 page-faults # 28.971 /sec
+34,606,498,021 cycles # 4.321 GHz
+25,120,741,129 instructions # 0.73 insn per cycle
+ 5,028,578,807 branches # 627.936 M/sec
+ 24,402,161 branch-misses # 0.49% of all branches
+We're doing a lot more context switches with domains, as expected due to the sleeps, +and we're executing many fewer instructions, which isn't surprising. +Reporting the counts for individual CPUs gets more interesting though:
+$ sudo perf stat -I 1000 -e instructions -Aa
+# Processes
+ 1.000409485 CPU0 5,106,261,160 instructions
+ 1.000409485 CPU1 2,746,012,554 instructions
+ 1.000409485 CPU2 14,235,084,764 instructions
+ 1.000409485 CPU3 7,545,940,906 instructions
+ 1.000409485 CPU4 2,605,655,333 instructions
+ 1.000409485 CPU5 6,023,131,238 instructions
+ 1.000409485 CPU6 2,860,656,865 instructions
+ 1.000409485 CPU7 8,195,416,048 instructions
+ 2.001406580 CPU0 5,674,686,033 instructions
+ 2.001406580 CPU1 2,774,756,912 instructions
+ 2.001406580 CPU2 12,231,014,682 instructions
+ 2.001406580 CPU3 8,292,824,909 instructions
+ 2.001406580 CPU4 2,592,461,540 instructions
+ 2.001406580 CPU5 7,182,922,668 instructions
+ 2.001406580 CPU6 2,742,731,223 instructions
+ 2.001406580 CPU7 7,219,186,119 instructions
+ 3.002394302 CPU0 4,676,179,731 instructions
+ 3.002394302 CPU1 2,773,345,921 instructions
+ 3.002394302 CPU2 13,236,080,365 instructions
+ 3.002394302 CPU3 5,142,640,767 instructions
+ 3.002394302 CPU4 2,580,401,766 instructions
+ 3.002394302 CPU5 13,600,129,246 instructions
+ 3.002394302 CPU6 2,667,830,277 instructions
+ 3.002394302 CPU7 4,908,168,984 instructions
+
+$ sudo perf stat -I 1000 -e instructions -Aa
+# Domains
+ 1.002680009 CPU0 3,134,933,139 instructions
+ 1.002680009 CPU1 3,140,191,650 instructions
+ 1.002680009 CPU2 3,155,579,241 instructions
+ 1.002680009 CPU3 3,059,035,269 instructions
+ 1.002680009 CPU4 3,102,718,089 instructions
+ 1.002680009 CPU5 3,027,660,263 instructions
+ 1.002680009 CPU6 3,167,151,483 instructions
+ 1.002680009 CPU7 3,214,267,081 instructions
+ 2.003692744 CPU0 3,009,806,420 instructions
+ 2.003692744 CPU1 3,015,194,636 instructions
+ 2.003692744 CPU2 3,093,562,866 instructions
+ 2.003692744 CPU3 3,005,546,617 instructions
+ 2.003692744 CPU4 3,067,126,726 instructions
+ 2.003692744 CPU5 3,042,259,123 instructions
+ 2.003692744 CPU6 3,073,514,980 instructions
+ 2.003692744 CPU7 3,158,786,841 instructions
+ 3.004694851 CPU0 3,069,604,047 instructions
+ 3.004694851 CPU1 3,063,976,761 instructions
+ 3.004694851 CPU2 3,116,761,158 instructions
+ 3.004694851 CPU3 3,045,677,304 instructions
+ 3.004694851 CPU4 3,101,053,228 instructions
+ 3.004694851 CPU5 2,973,005,489 instructions
+ 3.004694851 CPU6 3,109,177,113 instructions
+ 3.004694851 CPU7 3,158,349,130 instructions
+In the domains case all CPUs are doing roughly the same amount of work. +But when running separate processes the CPUs differ wildly! +Over the last 1-second interval, for example, CPU5 executed 5.3 times as many instructions as CPU4. +And indeed, some of the test processes are finishing much sooner than the others, +even though they all do the same work.
+Setting /sys/devices/system/cpu/cpufreq/policy*/energy_performance_preference to performance didn't make it faster,
+but setting it to power (power-saving mode) did make the processes benchmark much slower,
+while having little effect on the domains case!
So I think what's happening here with separate processes is that +the CPU is boosting the performance of one or two cores at a time, +allowing them to make lots of progress.
+But with domains this doesn't happen, either because no domain runs long enough before sleeping to trigger the boost, +or because as soon as it does it needs to stop and wait for the other domains for a GC and loses it.
+The main profiling and tracing tools used were:
+perf to take samples of CPU use, find hot functions and hot instructions within them,
+record process scheduling, look at hardware counters, and find sources of cache contention.
+statmemprof to find the source of allocations.
+eio-trace to visualise GC events and as a generic canvas for custom visualisations.
+magic-trace to see very detailed traces of recent activity when something goes wrong.
+olly to report on GC statistics.
+bpftrace for quick experiments about kernel behaviour.
+offcputime to see why a process is sleeping.
+I think OCaml 5's runtime events tracing was the star of the show here, making it much easier to see what was going on with GC,
+especially in combination with perf sched.
+statmemprof is also an essential tool for OCaml, and I'll be very glad to get it back with OCaml 5.3.
+I think I need to investigate perf more; I'd never used many of these features before.
+Though it is important to use it with offcputime etc to check you're not missing samples due to sleeping.
Unlike the previous post's example, where the cause was pretty obvious and led to a massive easy speed-up, +this one took a lot of investigation and revealed several problems, none of which seem very easy to fix. +I'm also a lot less confident that I really understand what's happening here, but here is a summary of my current guess:
+Since the sleeping mechanism will be changing in OCaml 5.3, +it would probably be worthwhile checking how that performs too. +I think there are some opportunities to improve the GC, such as letting idle domains opt out of GC after one collection, +and it looks like there are opportunities to reduce the amount of synchronisation done +(e.g. by letting late arrivers start the GC without having to wait for the leader, +or using a lock-free algorithm for becoming leader).
+For the solver, it would be good to try experimenting with CPU affinity to keep a subset of the 160 cores reserved for the solver. +Increasing the minor heap size and doing work in the main domain should also reduce the overhead of GC, +and improving the version compare function in the opam library would greatly reduce the need for it. +And if my goal was really to make it fast (rather than to improve multicore OCaml and its tooling) +then I'd probably switch it back to using processes.
+Finally, it was really useful that both of these blog posts examined performance regressions, +so I knew it must be possible to go faster. +Without a good idea of how fast something should be, it's easy to give up too early.
+Anyway, I hope you found some useful new tool in these posts!
+ diff --git a/data/planet/talex5/ocaml-5-performance-problems.md b/data/planet/talex5/ocaml-5-performance-problems.md new file mode 100644 index 0000000000..bbc36aa4a6 --- /dev/null +++ b/data/planet/talex5/ocaml-5-performance-problems.md @@ -0,0 +1,524 @@ +--- +title: OCaml 5 performance problems +description: +url: https://roscidus.com/blog/blog/2024/07/22/performance/ +date: 2024-07-22T10:00:00-00:00 +preview_image: +authors: +- Thomas Leonard +source: +--- + +Linux and OCaml provide a huge range of tools for investigating performance problems. +In this post I try using some of them to understand a network performance problem. +In part 2, I'll investigate a problem in a CPU-intensive multicore program.
+ +Table of Contents
+While porting capnp-rpc from Lwt to Eio, +to take advantage of OCaml 5's new effects system, +I tried running the benchmark to see if it got any faster:
+$ ./echo_bench.exe
+echo_bench.exe: [INFO] rate = 44933.359573 # The old Lwt version
+echo_bench.exe: [INFO] rate = 511.963565 # The (buggy) Eio version
+The benchmark records the number of echo RPCs per second. +Clearly, something is very wrong here! +In fact, the new version was so slow I had to reduce the number of iterations so it would finish.
+The old time command can immediately give us a hint:
$ /usr/bin/time ./echo_bench.exe
+1.85user 0.42system 0:02.31elapsed 98%CPU # Lwt
+0.16user 0.05system 0:01.95elapsed 11%CPU # Eio (buggy)
+(many shells provide their own time built-in with different output formats; I'm using /usr/bin/time here)
time's output shows time spent in user-mode (running the application's code on the CPU),
+time spent in the kernel, and the total wall-clock time.
+Both versions ran for around 2 seconds (doing a different number of iterations),
+but the Lwt version was using the CPU 98% of the time, while the Eio version was mostly sleeping.
eio-trace can be used to see what an Eio program is doing. +Tracing is always available (you don't need to recompile the program to get it).
+$ eio-trace run -- ./echo_bench.exe
+eio-trace run runs the command and displays the trace in a window.
+You can also use eio-trace record to save a trace and examine it later.
Trace of slow benchmark (12 concurrent requests)
The benchmark runs 12 test clients at once, making it a bit noisy. +To simplify thing, I set it to run only one client:
+
Trace of slow benchmark (one request at a time)
I've zoomed the image to show the first four iterations. +The first is so quick it's not really visible, but the next three take about 40ms each. +The yellow regions labelled "suspend-domain" show when the program is sleeping, waiting for an event from Linux. +Each horizontal bar is a fiber (a light-weight thread). From top to bottom they are:
+This trace immediately raises a couple of questions:
+Why is there a 40ms delay in each iteration of the test loop?
+Why does the program briefly wake up in the middle of the first delay, do nothing, and return to sleep? +(notice the extra "suspend-domain" at the top)
+Zooming in on a section between the delays, let's see what it's doing when it's not sleeping:
+ +After a 40ms delay, the server's read fiber receives the next request (the running fiber is shown in green). +The read fiber spawns a fiber to handle the request, which finishes quickly, starts the next read, +and then the write fiber transmits the reply.
+The client's read fiber gets the reply, the write fiber outputs a message, then the application fiber runs +and another message is sent. +The server reads something (presumably the first message, though it happens after the client had sent both), +then there is another long 40ms delay, then (far off the right of the image) the pattern repeats.
+To get more context in the trace, +I configured +the logging library to write the (existing) debug-level log messages to the trace buffer too:
+ +Log messages tend to be a bit long for the trace display, so they overlap and you have to zoom right in to read them, +but they do help navigate. +With this, I can see that the first client write is "Send finish" and the second is "Calling Echo.ping".
+Looks like we're not buffering the output, so it's doing two separate writes rather than combining them. +That's a little inefficient, and if you've done much network programming, +you also probably already know why this might cause a 40ms delay, +but let's pretend we don't know so we can play with a few more tools...
+strace can be used to trace interactions between applications and the Linux kernel
+(-tt -T shows when each call was started and how long it took):
$ strace -tt -T ./echo_bench.exe
+...
+11:38:58.079200 write(2, "echo_bench.exe: [INFO] Accepting"..., 73) = 73 <0.000008>
+11:38:58.079253 io_uring_enter(4, 4, 0, 0, NULL, 8) = 4 <0.000032>
+11:38:58.079341 io_uring_enter(4, 2, 0, 0, NULL, 8) = 2 <0.000020>
+11:38:58.079408 io_uring_enter(4, 2, 0, 0, NULL, 8) = 2 <0.000021>
+11:38:58.079471 io_uring_enter(4, 2, 0, 0, NULL, 8) = 2 <0.000018>
+11:38:58.079525 io_uring_enter(4, 2, 0, 0, NULL, 8) = 2 <0.000019>
+11:38:58.079580 io_uring_enter(4, 2, 0, 0, NULL, 8) = 2 <0.000013>
+11:38:58.079611 io_uring_enter(4, 1, 0, 0, NULL, 8) = 1 <0.000009>
+11:38:58.079637 io_uring_enter(4, 0, 1, IORING_ENTER_GETEVENTS|IORING_ENTER_EXT_ARG, 0x7ffc1661a480, 24) = -1 ETIME (Timer expired) <0.018913>
+11:38:58.098669 futex(0x5584542b767c, FUTEX_WAKE_PRIVATE, 1) = 1 <0.000105>
+11:38:58.098889 futex(0x5584542b7690, FUTEX_WAKE_PRIVATE, 1) = 1 <0.000059>
+11:38:58.098976 io_uring_enter(4, 0, 1, IORING_ENTER_GETEVENTS, NULL, 8) = 0 <0.021355>
+On Linux, Eio defaults to using the io_uring mechanism for submitting work to the kernel.
+io_uring_enter(4, 2, 0, 0, NULL, 8) = 2 means we asked to submit 2 new operations to the ring on FD 4,
+and the kernel accepted them.
The call at 11:38:58.079637 timed out after 19ms.
+It then woke up some futexes and then waited again, getting woken up after a further 21ms (for a total of 40ms).
Futexes are used to coordinate between system threads.
+strace -f will follow all spawned threads (and processes), not just the main one:
$ strace -T -f ./echo_bench.exe
+...
+[pid 48451] newfstatat(AT_FDCWD, "/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=40, ...}, 0) = 0 <0.000011>
+...
+[pid 48451] futex(0x561def43296c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, FUTEX_BITSET_MATCH_ANY <unfinished ...>
+...
+[pid 48449] io_uring_enter(4, 0, 1, IORING_ENTER_GETEVENTS|IORING_ENTER_EXT_ARG, 0x7ffe1d5d1c90, 24) = -1 ETIME (Timer expired) <0.018899>
+[pid 48449] futex(0x561def43296c, FUTEX_WAKE_PRIVATE, 1) = 1 <0.000106>
+[pid 48451] <... futex resumed>) = 0 <0.019981>
+[pid 48449] io_uring_enter(4, 0, 1, IORING_ENTER_GETEVENTS, NULL, 8 <unfinished ...>
+...
+[pid 48451] exit(0) = ?
+[pid 48451] +++ exited with 0 +++
+[pid 48449] <... io_uring_enter resumed>) = 0 <0.021205>
+...
+The benchmark connects to "127.0.0.1" and Eio uses getaddrinfo to look up addresses (we can't use uring for this).
+Since getaddrinfo can block for a long time, Eio creates a new system thread (pid 48451) to handle it
+(we can guess this thread is doing name resolution because we see it read resolv.conf).
As creating system threads is a little slow, Eio keeps the thread around for a bit after it finishes in case it's needed again. +The timeout is when Eio decides that the thread isn't needed any longer and asks it to exit. +So this isn't relevant to our problem (and only happens on the first 40ms delay, since we don't look up any further addresses).
+However, strace doesn't tell us what the uring operations were, or their return values.
+One option is to switch to the posix backend (which is the default on Unix systems).
+In fact, it's a good idea with any performance problem to check if it still happens with a different backend:
$ EIO_BACKEND=posix strace -T -tt ./echo_bench.exe
+...
+11:53:52.935976 writev(7, [{iov_base="\0\0\0\0\4\0\0\0\0\0\0\0\1\0\1\0\4\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0"..., iov_len=40}], 1) = 40 <0.000170>
+11:53:52.936308 ppoll([{fd=-1}, {fd=-1}, {fd=-1}, {fd=-1}, {fd=4, events=POLLIN}, {fd=-1}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}], 9, {tv_sec=0, tv_nsec=0}, NULL, 8) = 1 ([{fd=8, revents=POLLIN}], left {tv_sec=0, tv_nsec=0}) <0.000044>
+11:53:52.936500 writev(7, [{iov_base="\0\0\0\0\20\0\0\0\0\0\0\0\1\0\1\0\2\0\0\0\0\0\0\0\0\0\0\0\3\0\3\0"..., iov_len=136}], 1) = 136 <0.000055>
+11:53:52.936831 readv(8, [{iov_base="\0\0\0\0\4\0\0\0\0\0\0\0\1\0\1\0\4\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0"..., iov_len=4096}], 1) = 40 <0.000056>
+11:53:52.937516 ppoll([{fd=-1}, {fd=-1}, {fd=-1}, {fd=-1}, {fd=4, events=POLLIN}, {fd=-1}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}], 9, NULL, NULL, 8) = 1 ([{fd=8, revents=POLLIN}]) <0.038972>
+11:53:52.977751 readv(8, [{iov_base="\0\0\0\0\20\0\0\0\0\0\0\0\1\0\1\0\2\0\0\0\0\0\0\0\0\0\0\0\3\0\3\0"..., iov_len=4096}], 1) = 136 <0.000398>
+(to reduce clutter, I removed calls that returned EAGAIN and ppoll calls that returned 0 ready descriptors)
The problem still occurs, and now we can see the two writes:
+revents=POLLIN).
+ppoll to await further data.
+ppoll says FD 8 is now ready, and the server reads the other 136 bytes.
+Alternatively, we can trace uring operations using bpftrace. +bpftrace is a little scripting language similar to awk, +except that instead of editing a stream of characters, +it live-patches the running Linux kernel. +Apparently this is safe to run in production +(and I haven't managed to crash my kernel with it yet).
+Here is a list of uring tracepoints we can probe:
+$ sudo bpftrace -l 'tracepoint:io_uring:*'
+tracepoint:io_uring:io_uring_complete
+tracepoint:io_uring:io_uring_cqe_overflow
+tracepoint:io_uring:io_uring_cqring_wait
+tracepoint:io_uring:io_uring_create
+tracepoint:io_uring:io_uring_defer
+tracepoint:io_uring:io_uring_fail_link
+tracepoint:io_uring:io_uring_file_get
+tracepoint:io_uring:io_uring_link
+tracepoint:io_uring:io_uring_local_work_run
+tracepoint:io_uring:io_uring_poll_arm
+tracepoint:io_uring:io_uring_queue_async_work
+tracepoint:io_uring:io_uring_register
+tracepoint:io_uring:io_uring_req_failed
+tracepoint:io_uring:io_uring_short_write
+tracepoint:io_uring:io_uring_submit_req
+tracepoint:io_uring:io_uring_task_add
+tracepoint:io_uring:io_uring_task_work_run
+io_uring_complete looks promising:
$ sudo bpftrace -vl tracepoint:io_uring:io_uring_complete
+tracepoint:io_uring:io_uring_complete
+ void * ctx
+ void * req
+ u64 user_data
+ int res
+ unsigned cflags
+ u64 extra1
+ u64 extra2
+Here's a script to print out the time, process, operation name and result for each completion:
+1 +2 +3 +4 +5 +6 +7 +8 +9 +10 +11 +12 +13 +14 +15 +16 +17 +18 + | |
$ sudo bpftrace uringtrace.bt
+Attaching 3 probes...
+...
+1743ms: echo_bench.exe: WRITE_FIXED 40
+1743ms: echo_bench.exe: READV 40
+1743ms: echo_bench.exe: WRITE_FIXED 136
+1783ms: echo_bench.exe: READV 136
+In this output, the order is slightly different: +we see the server's read get the 40 bytes before the client sends the rest, +but we still see the 40ms delay between the completion of the second write and the corresponding read. +The change in order is because we're seeing when the kernel knew the read was complete, +not when the application found out about it.
+An obvious step with any networking problem is the look at the packets going over the network. +tcpdump can be used to capture packets, which can be displayed on the console or in a GUI with wireshark.
+$ sudo tcpdump -n -ttttt -i lo
+...
+...041330 IP ...37640 > ...7000: Flags [P.], ..., length 40
+...081975 IP ...7000 > ...37640: Flags [.], ..., length 0
+...082005 IP ...37640 > ...7000: Flags [P.], ..., length 136
+...082071 IP ...7000 > ...37640: Flags [.], ..., length 0
+Here we see the client (on port 37640) sending 40 bytes to the server (port 7000), +and the server replying with an ACK (with no payload) 40ms later. +After getting the ACK, the client socket sends the remaining 136 bytes.
+Here we can see that while the application made the two writes in quick succession, +TCP waited before sending the second one. +Searching for "delayed ack 40ms" will turn up an explanation.
+ss displays socket statistics.
+ss -tin shows all TCP sockets (-t) with internals (-i):
$ ss -tin 'sport = 7000 or dport = 7000'
+State Recv-Q Send-Q Local Address:Port Peer Address:Port
+ESTAB 0 0 127.0.0.1:7000 127.0.0.1:56224
+ ato:40 lastsnd:34 lastrcv:34 lastack:34
+ESTAB 0 176 127.0.0.1:56224 127.0.0.1:7000
+ ato:40 lastsnd:34 lastrcv:34 lastack:34 unacked:1 notsent:136
+There's a lot of output here; I've removed the irrelevant bits.
+ato:40 says there's a 40ms timeout for "delay ack mode".
+lastsnd, etc, say that nothing had happened for 34ms when this information was collected.
+unacked and notsent aren't documented in the man-page,
+but I guess it means that the client (now port 56224) is waiting for 1 packet to be ack'd and has 136 bytes waiting until then.
The client socket still has both messages (176 bytes total) in its queue; +it can't forget about the first message until the server confirms receiving it, +since the client might need to send it again if it got lost.
+This doesn't quite lead us to the solution, though.
+offwaketime records why a program stopped using the CPU, and what caused it to resume:
+$ sudo offwaketime-bpfcc -f -p (pgrep echo_bench.exe) > wakes
+$ flamegraph.pl --colors=chain wakes > wakes.svg
+
Time spent suspended along with wakeup reason
offwaketime records a stack-trace when a process is suspended (shown at the bottom and going up)
+and pairs it with the stack-trace of the thread that caused it to be resumed (shown above it and going down).
The taller column on the right shows Eio being woken up due to TCP data being received from the network, +confirming that it was the TCP ACK that got things going again.
+The shorter column on the left was unexpected, and the [UNKNOWN] in the stack is annoying
+(probably C code compiled without frame pointers).
+gdb gets a better stack trace.
+It turned out to be OCaml's tick thread, which wakes every 50ms to prevent one sys-thread from hogging the CPU:
$ strace -T -e pselect6 -p (pgrep echo_bench.exe) -f
+strace: Process 20162 attached with 2 threads
+...
+[pid 20173] pselect6(0, NULL, NULL, NULL, {tv_sec=0, tv_nsec=50000000}, NULL) = 0 (Timeout) <0.050441>
+[pid 20173] pselect6(0, NULL, NULL, NULL, {tv_sec=0, tv_nsec=50000000}, NULL) = 0 (Timeout) <0.050318>
+Having multiple threads shown on the same diagram is a bit confusing.
+I should probably have used -t to focus only on the main one.
Also, note that when using profiling tools that record the OCaml stack, +it's useful to compile with frame pointers enabled. +To install e.g. OCaml 5.2.0 with frame pointers enabled, use:
+1
+ | |
magic-trace allows capturing a short trace of everything the CPUs were doing just before some event. +It uses Intel Processor Trace to have the CPU record all control flow changes (calls, branches, etc) to a ring-buffer, +with fairly low overhead (2% to 10%, due to extra memory bandwidth needed). +When something interesting happens, we save the buffer and use it to reconstruct the recent history.
+Normally we'd need to set up a trigger to grab the buffer at the right moment, +but since this program is mostly idle it doesn't record much +and I just attached at a random point and immediately pressed Ctrl-C to grab a snapshot and detach:
+1 +2 +3 +4 + | |
As before, we see 40ms periods of waiting, with bursts of activity between them:
+
Magic trace showing 40ms delays
The output is a bit messed up because magic-trace doesn't understand that there are multiple OCaml fibers here, +each with their own stack. It also doesn't seem to know that exceptions unwind the stack.
+In each 40ms column, Eio_posix.Flow.single_read (3rd line from top) tried to do a read
+with readv, which got EAGAIN and called Sched.next to switch to the next fiber.
+Since there was nothing left to run, the Eio scheduler called ppoll.
+Linux didn't have anything ready for this process,
+and called the schedule kernel function to switch to another process.
I recorded an eio-trace at the same time, to see the bigger picture. +Here's the eio-trace zoomed in to show the two client writes (just before the 40ms wait), +with the relevant bits of the magic-trace stack pasted below them:
+
Zoomed in on the two client writes, showing eio-trace and magic-trace output together
We can see the OCaml code calling writev, entering the kernel, tcp_write_xmit being called to handle it,
+writing the IP packet to the network and then, because this is the loopback interface, the network receive logic
+handling the packet too.
+The second call is much shorter; tcp_write_xmit returns quickly without sending anything.
Note: I used the eio_posix backend here so it's easier to correlate the kernel operations to the application calls
+(uring queues them up and runs them later).
+The uring-trace project should make this easier in future, but doesn't integrate with eio-trace yet.
Zooming in further, it's easy to see the difference between the two calls to tcp_write_xmit:
The start of the first tcp_write_xmit and the whole of the second
+Looking at the source for tcp_write_xmit,
+we finally find the magic word "nagle"!
1 +2 +3 +4 + | |
Having identified a load of interesting events
+I wrote summary-posix.bt, a bpftrace script to summarise them.
+This includes log messages written by the application (by tracing write calls to stderr),
+reads and writes on the sockets,
+and various probed kernel functions seen in the magic-trace output and when reading the kernel source.
The output is specialised to this application (for example, TCP segments sent to port 7000 +are displayed as "to server", while others are "to client"). +I think this is a useful way to double-check my understanding, and any fix:
+$ sudo bpftrace summary-posix.bt
+[...]
+844ms: server: got ping request; sending reply
+844ms: server reads from socket (EAGAIN)
+844ms: server: writev(96 bytes)
+844ms: tcp_write_xmit (to client, nagle-on, packets_out=0)
+844ms: tcp_v4_send_check: sending 96 bytes to client
+844ms: tcp_v4_rcv: got 96 bytes
+844ms: timer_start (tcp_delack_timer, 40 ms)
+844ms: client reads 96 bytes from socket
+844ms: client: enqueue finish message
+844ms: client: enqueue ping call
+844ms: client reads from socket (EAGAIN)
+844ms: client: writev(40 bytes)
+844ms: tcp_write_xmit (to server, nagle-on, packets_out=0)
+844ms: tcp_v4_send_check: sending 40 bytes to server
+845ms: tcp_v4_rcv: got 40 bytes
+845ms: timer_start (tcp_delack_timer, 40 ms)
+845ms: client: writev(136 bytes)
+845ms: tcp_write_xmit (to server, nagle-on, packets_out=1)
+845ms: server reads 40 bytes from socket
+845ms: server reads from socket (EAGAIN)
+885ms: tcp_delack_timer_handler (ACK to client)
+885ms: tcp_v4_send_check: sending 0 bytes to client
+885ms: tcp_delack_timer_handler (ACK to server)
+885ms: tcp_v4_rcv: got 0 bytes
+885ms: tcp_write_xmit (to server, nagle-on, packets_out=0)
+885ms: tcp_v4_send_check: sending 136 bytes to server
+packets_out=0) so it gets sent immediately.
+packets_out=0) so the kernel sends it immediately.
+The problem seemed clear: while porting from Lwt to Eio I'd lost the output buffering. +So I looked at the Lwt code to see how it did it and... it doesn't! So how was it working?
+As I did with Eio, I set the Lwt benchmark's concurrency to 1 to simplify it for tracing, +and discovered that Lwt with 1 client thread has exactly the same problem as the Eio version. +Well, that's embarrassing! +But why is Lwt fast with 12 client threads?
+With only minor changes (e.g. write vs writev), the summary script above also worked for tracing the Lwt version.
+With 1 or 2 client threads, Lwt is slow, but with 3 it's fairly fast.
+The delay only happens if the client sends a "finish" message when the server has no replies queued up
+(otherwise the finish message unblocks the replies, which carry the ACK to the client immediately).
+So, it works mostly by fluke!
+Lwt just happens to schedule the threads in such a way that Nagle's algorithm mostly doesn't trigger with 12 concurrent requests.
Anyway, adding buffering to the Eio version fixed the problem:
+ +An interesting thing to notice here is that not only did the long delay go away,
+but the CPU operations while it was active were faster too!
+I think the reason is that the CPU goes into power-saving mode during the long delays.
+cpupower monitor shows my CPUs running at around 1 GHz with the old code and
+around 4.7 GHz when running the new version.
Here are the results for the fixed version:
+
$ ./echo_bench.exe
+echo_bench.exe: [INFO] rate = 44425.962625 # The old Lwt version
+echo_bench.exe: [INFO] rate = 59653.451934 # The fixed Eio version
+60k RPC requests per second doesn't seem that impressive, but at least it's faster than the old version, +which is good enough for now! There's clearly scope for improvement here (for example, the buffering I +added is quite inefficient, making two extra copies of every message, as the framing library copies it from +a cstruct to a string, and then I have to copy the string back to a cstruct for the kernel).
+There are lots of great tools available to help understand why something is running slowly (or misbehaving), +and since programmers usually don't have much time for profiling, +a little investigation will often turn up something interesting! +Even when things are working correctly, these tools are a good way to learn more about how things work.
+time will quickly tell you if the program is taking lots of time in application code, in the kernel, or just sleeping.
+If the problem is sleeping, offcputime and offwaketime can tell you why it was waiting and what woke it in the end.
+My own eio-trace tool will give a quick visual overview of what an Eio application is doing.
+strace is great for tracing interactions between applications and the kernel,
+but it doesn't help much when the application is using uring.
+To fix that, you can either switch to the eio_posix backend or use bpftrace with the uring tracepoints.
+tcpdump, wireshark and ss are all useful to examine network problems specifically.
I've found bpftrace to be really useful for all kinds of tasks.
+Being able to write quick one-liners or short scripts gives it great flexibility.
+Since the scripts run in the kernel you can also filter and aggregate data efficiently
+without having to pass it all to userspace, and you can examine any kernel data structures.
+We didn't need that here because the program was running so slowly, but it's great for many problems.
+In addition to using well-defined tracepoints,
+it can also probe any (non-inlined) function in the kernel or the application.
+I also think using it to create a "summary script" to confirm a problem and its solution seems useful,
+though this is the first time I've tried doing that.
magic-trace is great for getting really detailed function-by-function tracing through the application and kernel.
+Its ability to report the last few ms of activity after you notice a problem is extremely useful
+(though not needed in this example).
+It would be really useful if you could trigger magic-trace from a bpftrace script, but I didn't see a way to do that.
However, it was surprisingly difficult to get any of the tools to point directly +at the combination of Nagle's algorithm with delayed ACKs as the cause of this common problem!
+This post was mainly focused on what was happening in the kernel. +In part 2, I'll investigate a CPU-intensive problem instead.
+ diff --git a/data/planet/tarides/creating-the-syntaxdocumentation-command---part-3-vscode-platform-extension.md b/data/planet/tarides/creating-the-syntaxdocumentation-command---part-3-vscode-platform-extension.md new file mode 100644 index 0000000000..7c315744e4 --- /dev/null +++ b/data/planet/tarides/creating-the-syntaxdocumentation-command---part-3-vscode-platform-extension.md @@ -0,0 +1,201 @@ +--- +title: 'Creating the SyntaxDocumentation Command - Part 3: VSCode Platform Extension' +description: "In the final installment of our series on the SyntaxDocumentation command, + we delve into its integration within the OCaml VSCode Platform\u2026" +url: https://tarides.com/blog/2024-07-24-creating-the-syntaxdocumentation-command-part-3-vscode-platform-extension +date: 2024-07-24T00:00:00-00:00 +preview_image: https://tarides.com/static/aa5bd16e724bfc18f6e436399a4dda66/e49a8/vscode_toggle.jpg +authors: +- Tarides +source: +--- + +In the final installment of our series on the SyntaxDocumentation command, we delve into its integration within the OCaml VSCode Platform extension. Building on our previous discussions about Merlin and OCaml LSP, this article explores how to make SyntaxDocumentation an opt-in feature in the popular VSCode editor.
In the first part of this series, Creating the SyntaxDocumentation Command - Part 1: Merlin, we explored how to create a new command in Merlin, particularly the SyntaxDocumentation command. In the second part, Creating the SyntaxDocumentation Command - Part 2: OCaml LSP, we looked at how to implement this feature in OCaml LSP in order to enable visual editors to trigger the command with actions such as hovering. In this third and final installment, you will learn how SyntaxDocumentation integrates into the OCaml VSCode Platform extension as an opt-in feature, enabling users to toggle it on/off depending on their preference.
Visual Studio Code is a free open-source, cross-platform code editor from Microsoft that is very popular among developers. +Some of its features include:
+The VSCode OCaml Platform extension enhances the development experience for OCaml programmers. It is itself written in the OCaml programming language using bindings to the VSCode API and then compiled into Javascript with js_of_ocaml. It provides language support features such as syntax-highlighting, go-to-definition, auto-completion, and type-on-hover. These key functionalities are powered by the OCaml Language Server (ocamllsp), which can be installed using popular package managers like opam and esy. Users can easily configure the extension to work with different sandbox environments, ensuring a tailored setup for various project needs. Additionally, the extension includes comprehensive settings and command options, making it very versatile for both beginner and advanced OCaml developers.
The OCaml Platform Extension for VSCode gives us a nice UI for interacting with OCaml-LSP. We can configure settings for the server as well as interact with switches, browse the AST, and many more features. Our main focus is on adding a checkbox that allows users to activate or deactivate SyntaxDocumentation in OCaml LSP's hover response. I limited this article's scope to just the files relevant in implementing this, while giving a brief tour of how the extension is built.
Every VSCode extension has a manifest file, package.json, at the root of the extension directory. The package.json contains a mix of Node.js fields, such as scripts and devDependencies, and VS Code specific fields, like publisher, activationEvents, and contributes.
+Our manifest file contains general information such as:
We also have commands that act as action events for our extension. These commands are used to perform a wide range of things, like navigating the AST, upgrading packages, deleting a switch, etc. +An example of a command to open the AST explorer is written as:
+{
+ "command": "ocaml.open-ast-explorer-to-the-side",
+ "category": "OCaml",
+ "title": "Open AST explorer"
+}For our case, enabling/disabling SyntaxDocumentation is a configuration setting for our language server, so we indicate this in the configurations section:
"ocaml.server.syntaxDocumentation": {
+ "type": "boolean",
+ "default": false,
+ "markdownDescription": "Enable/Disable syntax documentation"
+}The file extension_instance.ml handles the setup and configuration of various components of the OCaml VSCode extension and ensures that features like the language server and documentation are properly initialised. Its key functionalities are:
type t = {
+ mutable sandbox : Sandbox.t;
+ mutable repl : Terminal_sandbox.t option;
+ mutable ocaml_version : Ocaml_version.t option;
+ mutable lsp_client : (LanguageClient.t * Ocaml_lsp.t) option;
+ mutable documentation_server : Documentation_server.t option;
+ documentation_server_info : StatusBarItem.t;
+ sandbox_info : StatusBarItem.t;
+ ast_editor_state : Ast_editor_state.t;
+ mutable codelens : bool option;
+ mutable extended_hover : bool option;
+ mutable dune_diagnostics : bool option;
+ mutable syntax_documentation : bool option;
+}Interacting With the Language Server: This extension needs to interact with the OCaml language server (ocamllsp) to provide features like code completion, diagnostics, and other language-specific functionalities.
Documentation Server Management: The file includes functionality to start, stop, and manage the documentation server, which provides documentation lookup for installed OCaml packages.
+Handling Configuration: This extension allows users to configure settings such as code lens, extended hover, diagnostics, and syntax documentation. These settings are sent to the language server to adjust its behaviour accordingly. For SyntaxDocumentation, whenever the user toggles the checkbox, the server should set the correct configuration parameters. This is done mainly using two functions set_configuration and send_configuration.
...
+
+(* Set configuration *)
+let set_configuration t ~syntax_documentation =
+ t.syntax_documentation <- syntax_documentation;
+ match t.lsp_client with
+ | None -> ()
+ | Some (client, (_ : Ocaml_lsp.t)) ->
+ send_configuration ~syntax_documentation client
+......
+
+(* Send configuration *)
+let send_configuration ~syntax_documentation client =
+ let syntaxDocumentation =
+ Option.map syntax_documentation ~f:(fun enable ->
+ Ocaml_lsp.OcamllspSettingEnable.create ~enable)
+ in
+ let settings =
+ Ocaml_lsp.OcamllspSettings.create
+ ~syntaxDocumentation
+ in
+ let payload =
+ let settings =
+ LanguageClient.DidChangeConfiguration.create
+ ~settings:(Ocaml_lsp.OcamllspSettings.t_to_js settings)
+ ()
+ in
+ LanguageClient.DidChangeConfiguration.t_to_js settings
+ in
+ LanguageClient.sendNotification
+ client
+ "workspace/didChangeConfiguration"
+ payload
+
+...The ocaml_lsp.ml file ensures that ocamllsp is set up correctly and up to date. For SyntaxDocumentation, two important modules used from this file are: OcamllspSettingEnable and OcamllspSettings.
OcamllspSettingEnable defines an interface for enabling/disabling specific settings in ocamllsp.
...
+
+module OcamllspSettingEnable = struct
+ include Interface.Make ()
+ include
+ [%js:
+ val enable : t -> bool or_undefined [@@js.get]
+ val create : enable:bool -> t [@@js.builder]]
+end
+
+...The annotation [@@js.get] is a PPX used to bind OCaml functions to JavaScript property accessors. This allows OCaml code to interact seamlessly with JavaScript objects, accessing properties directly as if they were native OCaml fields, while [@@js.builder] facilitates the creation of JavaScript objects from OCaml functions. They both come from the LexFi/gen_js_api library.
OcamllspSettings aggregrates multiple OcamllspSettingEnable settings into a comprehensive settings interface for ocamllsp.
...
+module OcamllspSettings = struct
+ include Interface.Make ()
+ include
+ [%js:
+ val syntaxDocumentation : t ->
+ OcamllspSettingEnable.t or_undefined [@@js.get]
+
+ val create : ?syntaxDocumentation:OcamllspSettingEnable.t ->
+ unit -> t [@@js.builder]]
+
+ let create ~syntaxDocumentation = create ?syntaxDocumentation ()
+end
+...The file settings.ml provides a flexible way to manage workspace-specific settings, including:
codelens and SyntaxDocumentation...
+let create_setting ~scope ~key ~of_json ~to_json =
+ { scope; key; to_json; of_json }
+
+let server_syntaxDocumentation_setting =
+ create_setting
+ ~scope:ConfigurationTarget.Workspace
+ ~key:"ocaml.server.syntaxDocumentation"
+ ~of_json:Jsonoo.Decode.bool
+ ~to_json:Jsonoo.Encode.bool
+...The vscode_ocaml_platform.ml file initialises and activates the OCaml Platform extension for VSCode. The key tasks include:
In the context of SyntaxDocumentation, this code ensures that the extension is correctly configured to handle SyntaxDocumentation settings. The notify_configuration_changes function listens for changes to the server_syntaxDocumentation_setting and updates the extension instance accordingly. This means that any changes the user makes to the SyntaxDocumentation settings in the VSCode workspace configuration will be reflected in the extension's behaviour, ensuring that SyntaxDocumentation is enabled or disabled as per the user's preference.
let notify_configuration_changes instance =
+ Workspace.onDidChangeConfiguration
+ ~listener:(fun _event ->
+ let syntax_documentation =
+ Settings.(get server_syntaxDocumentation_setting)
+ in
+ Extension_instance.set_configuration instance ~syntax_documentation)
+ ()
+
+
+ 
+
+
In this final article, we explored how to integrate SyntaxDocumentation into OCaml VSCode Platform extension as a configurable option for OCaml LSP's hover command. We covered key components such as configuring the extension manifest, managing the extension state, interacting with the OCaml language server, and handling workspace configurations. By enabling users to toggle the SyntaxDocumentation feature on or off, we can ensure a flexible and customisable development experience for all users.
Feel free to contribute to this extension on the GitHub repository: vscode-ocaml-platform. Thank you for following along in this series, and happy coding with OCaml and VSCode!
+diff --git a/data/planet/tarides/deep-dive-optimising-multicore-ocaml-for-windows.md b/data/planet/tarides/deep-dive-optimising-multicore-ocaml-for-windows.md new file mode 100644 index 0000000000..53b117e858 --- /dev/null +++ b/data/planet/tarides/deep-dive-optimising-multicore-ocaml-for-windows.md @@ -0,0 +1,60 @@ +--- +title: 'Deep Dive: Optimising Multicore OCaml for Windows' +description: "We love hosting internships. It is rewarding to potentially facilitate + someone\u2019s first foray into the OCaml ecosystem, helping them\u2026" +url: https://tarides.com/blog/2024-07-10-deep-dive-optimising-multicore-ocaml-for-windows +date: 2024-07-10T00:00:00-00:00 +preview_image: https://tarides.com/static/18316ffead39c18231bc3dd3899eed4f/6b50e/racecar.jpg +authors: +- Tarides +source: +--- + +Tarides is an open-source company first. Our top priorities are to establish and tend to the OCaml community. Similarly, we’re dedicated to the development of the OCaml language and enjoy collaborating with industry partners and individual engineers to continue improving the performance and features of OCaml.
+We want you to join the OCaml community, test the languages and tools, and actively be part of the language’s evolution.
+Contact Tarides to see how OCaml can benefit your business and/or for support while learning OCaml. Follow us on Twitter and LinkedIn to ensure you never miss a post, and join the OCaml discussion on Discuss!
+
We love hosting internships. It is rewarding to potentially facilitate someone’s first foray into the OCaml ecosystem, helping them establish a hopefully life-long foothold in the world of open-source programming. It is also a great opportunity to get new perspectives on existing issues. Fresh eyes can reinvigorate topics, highlighting different pain points and new solutions which benefit the entire community.
+Sometimes, we also find ourselves just trying to keep up with our interns as they take off like rocket ships! Recently, we mentored a student who did just that. The initial goal of the internship was to investigate strange performance drops in the OCaml runtime that arose after the introduction of multicore support. These performance drops were most keenly felt on Windows machines, and the initial internship specification emphasised the need to improve the developer experience on that operating system.
+Our intern @eutro went above and beyond anything we could have expected and tackled the project thoroughly and ambitiously. In this post, I will attempt to give you a comprehensive overview of this intricate project and the problems it tackled.
+Before OCaml 5, only one thread would run at any given time. Users never had to worry about multiple threads trying to use a shared resource like the Garbage Collector (GC). In OCaml 5, however, the process is divided into several 'threads'1, and multiple threads regularly try to run parts of the GC simultaneously. The minor GC uses a Stop The World (STW) function to run in parallel on all threads, whereas the major GC’s work is split into slices. These may happen in parallel between threads and while the user’s program (also called the ‘mutator’) is making changes. This is one example of when a mechanism is needed to protect multiple threads from making changes that contradict each other and result in unexpected behaviours.
+Locks are the traditional way of doing this, whereby other activity is halted (or locked) while one activity finishes. However, in multicore programming, this method would be incredibly inefficient since there can be many activities in progress simultaneously. In this case, we would need to introduce so many locks for the different parts of memory that doing so would cause memory and OS resource problems!
+The approach we use for OCaml 5 combines a Compare And Swap (CAS) operation with Busy-Wait loops. A CAS operation ensures that if two threads try to modify the same area of memory, only one will succeed. The one that fails will know it has failed and can then enter a period of Busy-Waiting (called SPIN_WAIT in the code). Busy-wait loops (also referred to as spins) describe a process that repeatedly ('busily') checks whether a condition is true. The process or task is only resumed once that condition is met.
Busy-wait loops are used successfully in OCaml for many purposes but have been optimised. They are mostly appropriate in cases where we think that the required condition will be met quickly or in a reasonable period of time. If that’s not the case, then theoretically, the thread that is waiting will just keep spinning. If one allows busy-wait loops to spin indefinitely, they waste a lot of power and CPU and can actually prevent the condition they are waiting for from being met. To avoid that happening, we can use a sleep function.
In order to implement spinning without wasting power, the loop checks the condition repeatedly, but after a while, it starts 'sleeping' between checks. Suppose a thread is waiting for condition C to come true, and it uses a Busy-Wait loop to check for this. The program spins a number of times, checking the condition, and then waits or goes to ‘sleep’ for a set amount of time – then it ‘wakes up’ and checks once more before (if it has to) going back to ‘sleep‘ again. The period of ‘sleep’ increases each time. This cycle repeats itself until the condition C finally comes true.
This was how the process was supposed to work, yet, for some unknown reason, certain processes would occasionally take much longer than expected. The performance drop was worst on Windows machines.
+The first order of business was to conduct a series of tests on the runtime. Not only to discover the possible cause of the performance drops but also to establish a baseline of performance against which to measure any changes (and hopefully improvements!).
+We knew that there was a performance problem and that it was particularly painful on Windows, but we didn’t know why. Even if we had a hunch as to what might be causing it, it was crucial to build up a realistic image of what was happening before we attempted a fix.
+@eutro began this process by identifying where Busy-Wait loops were spinning in the runtime and for how long. She also wanted to know if there were places in the runtime where processes would get ‘stuck’ in Busy-Wait loops and not move on, and if so, where and why.
+She used the OCaml testsuite and measured how many SPIN_WAIT macros resolved successfully without needing to sleep and which ones did not. She discovered that in most cases, the spinning had the desired effect, and the process could continue after a reasonable amount of time when the condition it was waiting for was met. The garbage collector was also not experiencing any significant performance drops, so it could not be the cause of the problems on Windows. Instead, what she realised was that on Windows, sleeps cannot be shorter than one millisecond, and so the first sleeps that occur end up being much too long. This causes extended and unnecessary delays for processes running on Windows. Equipped with this realisation, @eutro got started on a solution. One that would be most helpful on Windows but still benefit users on other operating systems.
There are a few ways a thread in OCaml can wait for a condition:
+So what has changed? As things stood, only steps one and two were available, a series of increasingly long sleeps interleaved with checks. So you would spin n times, then sleep for 10µs (‘µs’ is short for microseconds), then you check the condition once more and might sleep for 20µs, then 35µs, and so on. The point is that the time spent sleeping kept gradually increasing.
+However, as @eutro discovered, in many cases, the process took far too long to resume, even after the condition had come true. By the time they woke up from sleeping, they could have already proceeded if they had just ‘taken a ticket’ earlier and waited until they were notified. To improve performance, instead of repeatedly sleeping for longer increments, we use specialised ‘barriers’ to wait until we can proceed.
+To solve the Windows problem, we now use the SPIN_WAIT function only in cases where we don’t expect to ever need to sleep. In cases where that first sleep would cause significant delay, we introduce a new SPIN_WAIT_NTIMES function, which lets the process spin for a set number of times before being combined with a barrier. @eutro used her previous benchmarks to determine which occasions could keep the SPIN_WAIT cycle as-is and which occasions required the new SPIN_WAIT_NTIMES combined with a barrier.
But things didn’t stop there! @eutro could also optimise the type of barrier. Traditionally, we use condition variables to wake up threads waiting on a condition. However, they are unnecessarily resource-intensive as they require extra memory, and since woken threads must acquire (and release) a lock before they continue. A futex is a lower-level synchronisation primitive that can similarly be used to wake up threads but without the added complexity of a condition variable.
+@eutro added the use of futexes to the operating systems that permitted her to do so. Notably, macOS does not allow non-OS programs to use futexes so we fall back to using "condition variables" there.
+By introducing the use of SPIN_WAIT_NTIMES, barriers, and futexes, @eutro implemented a number of optimisations that were applicable not only on Windows but on several operating systems. These optimisations save users time and processing power.
During the course of implementing these changes, @eutro did a lot of tests. It was important to be thorough in order to ensure that her changes didn’t have unintended consequences. It is incredibly difficult to reason about how programs will react to a specific alteration, as there are many things happening in the runtime and several ways that programs can interact.
+She used the OCaml test suite again, this time to help her check that the OCaml runtime and other OCaml programs functioned correctly. Having verified that they were, @eutro also ran several benchmarks to check that she hadn’t actually made anything slower. For this, she used the Sandmark test suite.
+I recommend checking out the tests and benchmarks for yourself in the Pull Request. The PR also gives a more in-depth technical overview of the changes to the Busy-Waiting loops.
+It is great to see what someone with a passion for OCaml can bring to the system as a whole. I think it illustrates the benefits of open-source software development: when we invite fresh observations and suggestions, we create a community that supports innovation and collaboration. We are impressed with the hard work @eutro put into solving the complicated problem before her. She went above and beyond what we thought possible in such a short amount of time!
+Would you like to complete an internship with us? We welcome people of varying experience levels – some interns have made open-source contributions before and are familiar with functional programming, and some of our interns have no functional programming experience at all! If you’re interested in developing your skills in a supportive environment, keep an eye on our careers page, where we post about any available internships. We also regularly communicate about available internships on X (formerly known as Twitter). We hope to hear from you!
+++Tarides champions open-source development. We create and maintain key features of the OCaml language in collaboration with the OCaml community. To learn more about how you can support our open-source work, discover our page on GitHub.
+
++We are always happy to discuss commercial opportunities around OCaml. We provide core services, including training, tailor-made tools, and secure solutions. Contact us today to learn more about how Tarides can help your teams realise their vision.
+
It might be tempting to think that we can write code that works perfectly the first time around, but in reality optimisation and troubleshooting forms a big part of programming. However, there are more and less productive (and frustrating!) ways of problem solving. Having the right tools to guide you, ones that show you where to look and what is going wrong, can make a huge difference.
+We recently introduced you to the monitoring system runtime_events, which allows users to monitor their runtime for, among other things, how programs are affecting performance. Alongside runtime_events, sits the observability tool olly, which provides users with a number of helpful formatting options for their runtime tracing data.
This is all part of how we’re making developing in OCaml easier by bringing new features and tools to the community. Olly is just one such tool, and it makes the monitoring system for OCaml significantly more accessible. With Olly, you don’t have to be an expert or spend time combing through the data that runtime_events extracts for you. Rather, Olly can generate the information you need in a way that makes it easy to understand, store, and query.
Olly, as an observability tool for OCaml 5, has the ability to extract runtime tracing data from runtime_events. This data can then be visualised with a variety of graphical options available.
How does Olly do this? Olly uses the Runtime API to provide you with monitoring metric information and associated data. The tool comes with several subcommands, each with its own function.
+The command olly trace can generate runtime traces for programs compiled in OCaml 5 using its trace subcommand. The tracing data is generated in one of two formats, the Fuschia trace format or the Chrome tracing format with the former being the default. Both formats can be viewed in Perfetto, but the Chrome format trace can also be viewed in chrome://tracing for Chromium-based browsers. Another example of a subcommand is olly gc-stats, which can report the running time of the garbage collector (GC) and the GC tail latency of an OCaml executable.
The motivation behind introducing an observability tool like Olly is to make data extracted using runtime_events more useful, since few developers will want to use the event tracing system directly. Olly makes it easy for users to troubleshoot their own programs, but it also makes it easy for a developer to diagnose why someone else’s program is slow. A client can send their runtime_events data, a feature that comes built in with every OCaml 5 switch, to a developer who can then use Olly to find the problem and suggest a solution. This makes working in OCaml is easier as optimisation and problem solving becomes more efficient and streamlined.
It doesn’t end there! One of our future goals for Olly is that it should be able to provide automatic reports and diagnosis of some problems. Look out for that exciting update in the future!
+One of the latest updates to Olly is its modularisation by Eutro, splitting the bin/olly.ml file into smaller discrete libraries including olly_common, olly_trace, and olly_gc_stats. By splitting up the large file, the user can exercise some control over which dependencies they want their library to have. They can create a minimal build with minimal dependencies, or stick with a fuller build relying on all the dependencies. For example, to build olly_bare on the trunk you now only require two dependencies: Dune and cmdliner. Both can be installed without using Opam. Since some developers will prefer this set up, it’s good to support a variety of configurations.
It also potentially makes it easier to maintain, since the smaller files have well-defined purposes and provide a clearer overview than just having one large file covering a multitude of functions. If something breaks, this segmentation can make it easier for a maintainer to triage and amend the problem. The same modularisation may also help newcomers get an overview of all the different components of the library. Sadiq Jaffer merged Eutro’s PR #43 into Tarides: main and it will form part of a future Olly release pending further testing.
Let's wrap up by looking at an example of when you might use Olly. When we want to visualise the performance of the OCaml runtime alongside any custom events we may have, the first step is to generate a trace. To generate a trace, we run the command olly trace tracefile.trace in combination with the name of the program we want to enable tracing for. If we wanted to generate a trace for the solver-service, the command would be olly trace tracefile.trace 'solver-service'.
For our example, we chose to generate the tracing data in the Fuschia trace format. Once we had the trace, we loaded it into Perfetto to get a helpful visual representation of what our code is doing and we ended up with the following image:
+
+
+
+ 
+
+
The UI in this image displays the processes down the side, each corresponding to a domain. Our program ended up using four cores, and therefore, the image shows four processes. Each process, in turn, shows the tracing for the OCaml runtime build plus the custom events generated by Eio. Let's zoom in on one process now:
+
+
+
+ 
+
+
This expanded view shows both the Garbage Collector's (GC) activity and times when Eio is suspended.
+We want to create tools that make the developer experience in OCaml easier and more intuitive. Olly makes it possible to visualise your code's performance, helping you understand when your programs are slowing down and why. If you have suggestions or improvements to share, you are welcome to participate in the Runtime Events Tools repo on GitHub.
+We want to hear from you! Connect with us on social media by following us on X (formerly known as Twitter) and LinkedIn. You can also join in with the rest of the community on the forum Discuss to share your thoughts on everything OCaml!
+++Tarides champions open-source development. We create and maintain key features of the OCaml language in collaboration with the OCaml community. To learn more about how you can support our open-source work, discover our page on GitHub.
+
+diff --git a/data/planet/tarides/ocaml-compiler-manual-html-generation.md b/data/planet/tarides/ocaml-compiler-manual-html-generation.md new file mode 100644 index 0000000000..c60716a67e --- /dev/null +++ b/data/planet/tarides/ocaml-compiler-manual-html-generation.md @@ -0,0 +1,109 @@ +--- +title: OCaml Compiler Manual HTML Generation +description: "In order to avoid long, confusing URLs on the OCaml Manual pages, we + set out to create a solution that shortens these URLs, including\u2026" +url: https://tarides.com/blog/2024-07-17-ocaml-compiler-manual-html-generation +date: 2024-07-17T00:00:00-00:00 +preview_image: https://tarides.com/static/71ebe0c7b3ff03df0f8cfbf681e8dad8/0132d/compiler-manual.jpg +authors: +- Tarides +source: +--- + +We are always happy to discuss commercial opportunities around OCaml. We provide core services, including training, tailor-made tools, and secure solutions. Contact us today to learn more about how Tarides can help your teams realise their vision.
+
In order to avoid long, confusing URLs on the OCaml Manual pages, we set out to create a solution that shortens these URLs, including section references, and contains the specific version. The result improves readability and user experience. This article outlines the motivation behind these changes and how we implemented them.
+The OCaml HTML manuals have URL references such as https://v2.ocaml.org/manual/types.html#sss:typexpr-sharp-types, and they do not refer to any specific compiler version. We needed a way to easily share a link with the version number included. The OCaml.org page already has a mention of the compiler version, but it refers to specific https://ocaml.org/releases.
+We wanted a canonical naming convention that is consistent with current and future manual releases. It would also be beneficial to have only one place to store all the manuals, and the users of OCaml.org should never see redirecting URLs in the browser. This will greatly help increase the overall backlink quality when people share the links in conversations, tutorials, blogs, and on the Web. A preferred naming scheme should be something like:
+https://v2.ocaml.org/releases/latest/manual/attributes.html +https://v2.ocaml.org/releases/4.12/manual/attributes.html
+Using this, we redirected the v2.ocaml.org to OCaml.org for the production deployment. Also, the changes help in shorter URLs that can be easily remembered and shared. The rel="canonical" is a perfectly good way to make sure only https://ocaml.org/manual/latest gets indexed.
+After a detailed discussion, the following UI mockup to switch manuals was provided via GitHub issue, and Option A was selected.
+
+
+
+ 
+
+
Our proposed changes to the URL are shown below:
+Current: https://v2.ocaml.org/releases/5.1/htmlman/index.html
+Suggested: https://ocaml.org/manual/5.3.0/index.html
Current: https://v2.ocaml.org/releases/5.1/api/Atomic.html
+Suggested: https://ocaml.org/manual/5.3.0/api/Atomic.html
The HTML manual files are hosted in a separate GitHub repository at https://github.com/ocaml-web/html-compiler-manuals/. It contains a folder for each compiler version, and it also has the manual HTML files.
+A script to automate the process of generating the HTML manuals is also available at https://github.com/ocaml-web/html-compiler-manuals/blob/main/scripts/build-compiler-html-manuals.sh. The script defines two variables, DIR and OCAML_VERSION, where you can specify the location to build the manual and the compiler version to use. It then clones the ocaml/ocaml repository, switches to the specific compiler branch, builds the compiler, and then generates the manuals. The actual commands are listed below for reference:
echo "Clone ocaml repository ..."
+git clone git@github.com:ocaml/ocaml.git
+
+# Switch to ocaml branch
+echo "Checkout $OCAML_VERSION branch in ocaml ..."
+cd ocaml
+git checkout $OCAML_VERSION
+
+# Remove any stale files
+echo "Running make clean"
+make clean
+git clean -f -x
+
+# Configure and build
+echo "Running configure and make ..."
+./configure
+make
+
+# Build web
+echo "Generating manuals ..."
+cd manual
+make webAs per the new API requirements, the manual/src/html_processing/Makefile variables are updated as follows:
WEBDIRMAN = $(WEDBIR)/$(VERSION)
+WEBDIRAPI = $(WEBDIRMAN)/APIAccordingly, we have also updated the manual/src/html_processing/src/common.ml.in file OCaml variables to reflect the required changes:
+let web_dir = Filename.parent_dir_name // "webman" // ocaml_version
+
+let docs_maindir = web_dir
+
+let api_page_url = "api"
+
+let manual_page_url = ".."We also include the https://plausible.ci.dev/js/script.js script to collect view metrics for the HTML pages. The manuals from 3.12 through 5.2 are now available in the https://github.com/ocaml-web/html-compiler-manuals/tree/main GitHub repository.
+The OCaml.org Dockerfile has a step included to clone the HTML manuals and perform an automated production deployment as shown below:
+RUN git clone https://github.com/ocaml-web/html-compiler-manuals /manual
+
+ENV OCAMLORG_MANUAL_PATH /manualThe path to the new GitHub repository has been updated in the configuration file, along with the explicit URL paths to the respective manuals. The v2 URLs from the data/releases/*.md file have been replaced without the v2 URLs, and the manual /releases/ redirects have been removed from redirection.ml. The /releases/ redirects are now handled in middleware.ml. The caddy configuration to allow the redirection of v2.ocaml.org can be implemented as follows:
v2.ocaml.org {
+ redir https://ocaml.org{uri} permanent
+}You are encouraged to checkout the latest OCaml compiler from trunk and use the build-compiler-html-manual.sh script to generate the HTML documentation.
Please do report any errors or issues that you face at the following GitHub repository: https://github.com/ocaml-web/html-compiler-manuals/issues
+If you are interested in working on OCaml.org, please message us on the OCaml Discord server or reach out to the contributors in GitHub.
+(cross-ref) Online OCaml Manual: there should be an easy way to get a fixed-version URL. https://github.com/ocaml/ocaml.org/issues/534
+Use webman/*.html and webman/api for OCaml.org manuals. https://github.com/ocaml/ocaml/pull/12976
Serve OCaml Compiler Manuals. https://github.com/ocaml/ocaml.org/pull/2150
+Simplify and extend /releases/ redirects from legacy v2.ocaml.org URLs. https://github.com/ocaml/ocaml.org/pull/2448
++Tarides is an open-source company first. Our top priorities are to establish and tend to the OCaml community. Similarly, we’re dedicated to the development of the OCaml language and enjoy collaborating with industry partners and individual engineers to continue improving the performance and features of OCaml. We want you to join the OCaml community, test the languages and tools, and actively be part of the language’s evolution.
+
+Tarides is also always happy to discuss commercial opportunities around OCaml. There are many areas where we can help industrial users to adopt OCaml 5 more quickly, including training, support, custom developments, etc. Please contact us if you are interested in discussing your needs.
+