Relationship with folly::coro & asynchronous programming

[Weaverzhu]

It seems that a lot of apis are quite the same with folly::coro. I'm a big fan of coroutines and asynchronous programming. What do you guys think of the relationship?

Would async_simples do more (e.g supports trace spans likes opentelemetry between suspend points, call stack traces likes folly::AsyncStack, latency stats). Some interesting topics (examples users may want to learn) are:

1. Custom awaitables so user can implement anything they want across suspend points.
2. Easy ways to migrate to asychronous programming (the cost is closed related to the num of function calls, much less than callback apis & future/promise, but still a little painful).
3. Library maintainers also need to provide two sets of apis if not all of their users decide to upgrade toolchains to use coroutines, is there anyway to ease the efforts?
4. In what real circumstances (call depth over hundreds does not make sense for real projects in benchmark) coroutine suspention (e.g. initial_suspend, allocation for new coroutine objects) may cause performance issues.

[ChuanqiXu9]

It seems that a lot of apis are quite the same with folly::coro. I'm a big fan of coroutines and asynchronous programming. What do you guys think of the relationship?

In the very beginning, we looked a lot the implementation from folly and we explained this in document. The reason why we don't use folly directly is that we feel folly it too heavy for us. Even for folly::coro, there are features we don't use. And the features are not free even if we don't use it. Also we feel it is unnecessarily complex for us to code completely in folly style.

Would async_simples do more (e.g supports trace spans likes opentelemetry between suspend points, call stack traces likes folly::AsyncStack, latency stats).

No, possibly we won't. On the one hand, folly doesn't have the features when we started to implement async_simple. On the other hand, such features have costs. For example, asyncStack will cause the switch performance of coroutines slow down about 3 times. Meanwhile, our solution to the question lives in the compiler side, see https://github.com/alibaba/async_simple/blob/main/docs/docs.en/DebuggingLazy.md. It is much more cheap than asyncStack.

Custom awaitables so user can implement anything they want across suspend points.

Yes. We support this. We can find the example in demos. Also we support to co_await a Future to make things simpler: https://github.com/alibaba/async_simple/blob/main/docs/docs.en/Future.md#get-value-from-lazy

Easy ways to migrate to asychronous programming (the cost is closed related to the num of function calls, much less than callback apis & future/promise, but still a little painful).

This is a complex topic. We can't talk it precisely and concisely. Maybe you can take a look at https://github.com/alibaba/async_simple/blob/main/docs/docs.en/StacklessCoroutineAndFuture.md.

Library maintainers also need to provide two sets of apis if not all of their users decide to upgrade toolchains to use coroutines, is there anyway to ease the efforts?

We can wrap the coroutine APIs into Futures. See the above doc.

In what real circumstances (call depth over hundreds does not make sense for real projects in benchmark) coroutine suspention (e.g. initial_suspend, allocation for new coroutine objects) may cause performance issues.

It is not easy to give a precise and concise answer too. But we can estimate the cost of coroutine switch into 2 function calls, which takes generally no more than 10ns.

[Weaverzhu]

Thanks for the quick answers!

We can wrap the coroutine APIs into Futures. See the above doc.

What about optimizing a complex library, eliminating all blocking calls? Shouldn't we also migrate to asynchronous programming with future/promise, which is more complex than async/await model. Otherwise users without latest gcc toolchain can only use precompiled library.

Also, is there a good way to quantitively analyze how eliminating blocking calls would improve the performance? Mentioned in readme it has been widely used in many production areas. How do you tell if asynchronous programming is worth efforts (e.g. toolchain upgrades, possible programming model migration)?

Meanwhile, our solution to the question lives in the compiler side, see https://github.com/alibaba/async_simple/blob/main/docs/docs.en/DebuggingLazy.md. It is much more cheap than asyncStack.

Very impressive, folly::coro is attractive to me because of the native support to FBThrift. But with coroutine APIs the many asychronous solutions can be mixed together. (co_await a folly::Future as long as there is a FollyFutureAwaitable or await_transform). On the other hand folly is not a very 'pleasing' library, some of the key APIs are breaking during the upgrade, which is really annoying, and the dependencies may be too much. I think I'll take some time and look into this library.

[ChuanqiXu9]

What about optimizing a complex library, eliminating all blocking calls?

This requires a refactoring. I don't think we have a magic solution for such problems.

Shouldn't we also migrate to asynchronous programming with future/promise, which is more complex than async/await model. Otherwise users without latest gcc toolchain can only use precompiled library.

I mean to refactor the library with coroutines. But in the headers, we can offer an option to only export the future wrapped APIs.

Very impressive, folly::coro is attractive to me because of the native support to FBThrift. But with coroutine APIs the many asychronous solutions can be mixed together. (co_await a folly::Future as long as there is a FollyFutureAwaitable or await_transform). On the other hand folly is not a very 'pleasing' library, some of the key APIs are breaking during the upgrade, which is really annoying, and the dependencies may be too much. I think I'll take some time and look into this library.

Complexity is a cost too, including code complexity and architecture complexity. This is the major reason why we don't use folly.

[Weaverzhu]

Speaking of executors, the docs said "The executor used in alibaba actually isn't open-sourced yet.". Is there any plans to kindly provide an executor?

Folly executors do not use work-stealing, while bthread does. Would you share some ideas about how should we implement an executor for IO-CPU hybrid workload.

[ChuanqiXu9]

Speaking of executors, the docs said "The executor used in alibaba actually isn't open-sourced yet.". Is there any plans to kindly provide an executor?

The executor used internally is the product of another team. And they don't have plans to open source their codes yet. They published a paper about their system (http://www.vldb.org/pvldb/vol13/p3272-jiang.pdf) (not only about executors). You can look at it if you're interested.

Folly executors do not use work-stealing, while bthread does. Would you share some ideas about how should we implement an executor for IO-CPU hybrid workload.

It is not trivial to implement an efficiency executor. In our coroutines-based RPC library, we used ASIO as the executor (https://github.com/alibaba/yalantinglibs). You can look at it you're not intending to reinvent the wheels.

[4kangjc]

Speaking of executors, the docs said "The executor used in alibaba actually isn't open-sourced yet.". Is there any plans to kindly provide an executor?

The executor used internally is the product of another team. And they don't have plans to open source their codes yet. They published a paper about their system (http://www.vldb.org/pvldb/vol13/p3272-jiang.pdf) (not only about executors). You can look at it if you're interested.

Folly executors do not use work-stealing, while bthread does. Would you share some ideas about how should we implement an executor for IO-CPU hybrid workload.

It is not trivial to implement an efficiency executor. In our coroutines-based PRC library, we used ASIO as the executor (https://github.com/alibaba/yalantinglibs). You can look at it you're not intending to reinvent the wheels.

typo: PRC -> RPC

[Weaverzhu]

facebook/rocksdb#11017 mentioned a stackful coroutine PhotonLibOs. Stackful-coroutines are far less painful when rewriting code for asynchronous programming. On the other hand, It seems that folly stackless coroutines met some problem in rocksdb, through the main issues are related with file IO rather than network.

Is there any major difference between bthread, PhontonLibOs & async_simple coroutines?

[ChuanqiXu9]

async_simple offers uthread, which is a stackful coroutines too.

I didn't look into bthread, photonoslib and other stackful libraries very carefully. I don't feel there will be a significant differ ence in the implementations. Maybe there will some optimization like stackful coroutines pool just like thread pools.

For asynchronous programming, the most important part may the executors, which is not included in async_simple. We just try to make it configurable.

For the level of asyncronous component, the stackful coroutines looks not good at least in the perspective of tools. Since stackful coroutines are implemented by assembly, the tools can't know about it. So that the it maybe compiled and analysed incorrectly. And this may not to be solved since it is non standard.