Skip to content

Commit

Permalink
Merge pull request #51 from hit9/fix-compatible-text
Browse files Browse the repository at this point in the history
Fix that extensible is forward_compatibility (instead of backward)
  • Loading branch information
hit9 authored May 22, 2023
2 parents 88171fb + 29cf450 commit 5bf13d5
Show file tree
Hide file tree
Showing 11 changed files with 58 additions and 57 deletions.
4 changes: 2 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ Features
---------

- Supports bit level data serialization, **born for embedded development**.
- Supports protocol `extensiblity <https://bitproto.readthedocs.io/en/latest/language.html#extensibility>`_ , for backward-compatibility.
- Supports protocol `extensiblity <https://bitproto.readthedocs.io/en/latest/language.html#extensibility>`_ , for forward-compatibility.
- Easy to start, syntax is similar to the well-known protobuf.
- Supports languages: `C <https://bitproto.readthedocs.io/en/latest/c-guide.html>`_ (without dynamic memory allocation),
`Go <https://bitproto.readthedocs.io/en/latest/go-guide.html>`_, `Python <https://bitproto.readthedocs.io/en/latest/python-guide.html>`_ .
Expand Down Expand Up @@ -217,7 +217,7 @@ The differences between bitproto and protobuf are:
on structs in C.

* Protobuf works good on
`backward compatibility <https://developers.google.com/protocol-buffers/docs/overview#backwards_compatibility>`_.
`forward compatibility <https://developers.google.com/protocol-buffers/docs/overview#forward_compatibility>`_.
For bitproto, this is the main shortcome of bitproto serialization until v0.4.0, since this version, it supports message's
`extensiblity <https://bitproto.readthedocs.io/en/latest/language.html#extensibility>`_ by adding two bytes indicating
the message size at head of the message's encoded buffer. This breaks the
Expand Down
2 changes: 1 addition & 1 deletion changes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Version 0.4.2
Version 0.4.0
-------------

- Add support for ``message`` and ``enum`` extensiblity for protocol backward compatibility.
- Add support for ``message`` and ``enum`` extensiblity for protocol forward compatibility.
- Cut down the code size of generated language-specific files.
- Refactor the bitproto compiler.
- Refactor the bitproto serialization mechanism, using language-specific libraries instead of pure compiler-generated files.
2 changes: 1 addition & 1 deletion compiler/bitproto/_ast.py
Original file line number Diff line number Diff line change
Expand Up @@ -736,7 +736,7 @@ def __repr__(self) -> str:

@dataclass
class ExtensibleType(Type):
"""ExtensibleType is the type able to extend its size for backward-compatiable
"""ExtensibleType is the type able to extend its size for forward-compatiable
concern. When setting the `extensible` parameter to True, a micro buffer that
indicates the type size will be inserted at the head of encoded buffer.
In bitproto, array, enum and message are extensible types.
Expand Down
2 changes: 1 addition & 1 deletion compiler/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
---------
- Supports bit level data serialization.
- Supports protocol extensiblity, for backward-compatibility.
- Supports protocol extensiblity, for forward-compatibility.
- Very easy to start
- Protocol syntax is similar to the well-known protobuf.
Expand Down
6 changes: 2 additions & 4 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Features
---------

- Supports bit level data serialization, **born for embedded development**.
- Supports protocol :ref:`extensiblity <language-guide-extensibility>`, for backward-compatibility.
- Supports protocol :ref:`extensiblity <language-guide-extensibility>`, for forward-compatibility.
- Easy to start, syntax is similar to the well-known protobuf.
- Supports languages: - :ref:`C <quickstart-c-guide>` (without dynamic memory allocation), :ref:`Go <quickstart-go-guide>`, - :ref:`Python <quickstart-python-guide>`.
- Blazing fast encoding/decoding, :ref:`benchmark <performance-benchmark>`.
Expand Down Expand Up @@ -132,9 +132,7 @@ The differences between bitproto and protobuf are:
think setting `aligned attribute to 1 <https://stackoverflow.com/a/11772340>`_
on structs in C.

* Protobuf works good on
`backward compatibility <https://developers.google.com/protocol-buffers/docs/overview#backwards_compatibility>`_.
For bitproto, this is the main shortcome of bitproto serialization until
* Forward-compatibility is the main shortcome of bitproto serialization until
:ref:`v0.4.0 <version-0.4.0>`, since this version, it supports message's
:ref:`extensiblity <language-guide-extensibility>` by adding two bytes indicating
the message size at head of the message's encoded buffer. This breaks the
Expand Down
4 changes: 2 additions & 2 deletions docs/language.rst
Original file line number Diff line number Diff line change
Expand Up @@ -437,11 +437,11 @@ Extensibility
^^^^^^^^^^^^^

Bitproto knows exactly how many bits a message will occupy at compile time, because all types
are fix-sized. This may make backwards-compatibility hard.
are fix-sized. This may make `forward-compatibility <https://en.wikipedia.org/wiki/Forward_compatibility>`_ hard.

It seems ok to add new fields to the end of a message in use, because the structures of
existing fields are unchanged, the decoding end won't scan the encoded bytes of new fields,
then "the backward-compatibility achieved":
then "the forward-compatibility achieved":

.. sourcecode:: bitproto

Expand Down
7 changes: 3 additions & 4 deletions docs/locales/zh/LC_MESSAGES/changelog.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ msgid ""
msgstr ""
"Project-Id-Version: bitproto 0.4.0\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2023-01-02 22:16+0800\n"
"POT-Creation-Date: 2023-05-22 21:23+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.11.0\n"
"Generated-By: Babel 2.12.1\n"

#: ../../changelog.rst:2 eb5e577b5cca451f97c84e6697bf5395
msgid "Changelog"
Expand Down Expand Up @@ -94,7 +94,7 @@ msgstr ""
#: ../../../changes.rst:58 3c499990cd38434f932a3558376617c5
msgid ""
"Add support for ``message`` and ``enum`` extensiblity for protocol "
"backward compatibility."
"forward compatibility."
msgstr ""

#: ../../../changes.rst:59 ad10bcd6f05a4e5187b2d8ba2f3e5235
Expand All @@ -110,4 +110,3 @@ msgid ""
"Refactor the bitproto serialization mechanism, using language-specific "
"libraries instead of pure compiler-generated files."
msgstr ""

50 changes: 24 additions & 26 deletions docs/locales/zh/LC_MESSAGES/index.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ msgid ""
msgstr ""
"Project-Id-Version: bitproto 0.4.0\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-12-31 00:08+0800\n"
"POT-Creation-Date: 2023-05-22 21:26+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.11.0\n"
"Generated-By: Babel 2.12.1\n"

#: ../../index.rst:2 bdade4b4967c435aba586a8934eea740
msgid "The bit level data interchange format"
Expand Down Expand Up @@ -60,8 +60,8 @@ msgstr "支持比特级别的数据序列化, 为嵌入式开发而生"
#: ../../index.rst:48 2ff118718b4a40be88167d8b2d2f126d
msgid ""
"Supports protocol :ref:`extensiblity <language-guide-extensibility>`, for"
" backward-compatibility."
msgstr "支持协议的 :ref:`扩展性 <language-guide-extensibility>`、向后兼容设计"
" forward-compatibility."
msgstr "支持协议的 :ref:`扩展性 <language-guide-extensibility>`、向前兼容设计"

#: ../../index.rst:49 0de44305b7f64e5ea96263ca97b55b9a
msgid "Easy to start, syntax is similar to the well-known protobuf."
Expand All @@ -72,7 +72,9 @@ msgid ""
"Supports languages: - :ref:`C <quickstart-c-guide>` (without dynamic "
"memory allocation), :ref:`Go <quickstart-go-guide>`, - :ref:`Python "
"<quickstart-python-guide>`."
msgstr "支持的语言: :ref:`C (无动态内存申请) <quickstart-c-guide>`, :ref:`Go <quickstart-go-guide>` 和 :ref:`Python <quickstart-python-guide>`."
msgstr ""
"支持的语言: :ref:`C (无动态内存申请) <quickstart-c-guide>`, :ref:`Go <quickstart-go-"
"guide>` 和 :ref:`Python <quickstart-python-guide>`."

#: ../../index.rst:51 7cf4617bb795450faafb37d3271c17c5
msgid "Blazing fast encoding/decoding, :ref:`benchmark <performance-benchmark>`."
Expand Down Expand Up @@ -216,41 +218,37 @@ msgstr ""

#: ../../index.rst:135 3e018375744b48ce8a5b0c61e2ab8460
msgid ""
"Protobuf works good on `backward compatibility "
"<https://developers.google.com/protocol-"
"buffers/docs/overview#backwards_compatibility>`_. For bitproto, this is "
"the main shortcome of bitproto serialization until :ref:`v0.4.0 "
"<version-0.4.0>`, since this version, it supports message's "
":ref:`extensiblity <language-guide-extensibility>` by adding two bytes "
"indicating the message size at head of the message's encoded buffer. "
"This breaks the traditional data layout design by encoding some minimal "
"reflection size information in, so this is designed as an optional "
"feature."
"Forward-compatibility is the main shortcome of bitproto serialization "
"until :ref:`v0.4.0 <version-0.4.0>`, since this version, it supports "
"message's :ref:`extensiblity <language-guide-extensibility>` by adding "
"two bytes indicating the message size at head of the message's encoded "
"buffer. This breaks the traditional data layout design by encoding some "
"minimal reflection size information in, so this is designed as an "
"optional feature."
msgstr ""
"Protobuf 对于 `向后兼容 <https://developers.google.com/protocol-"
"buffers/docs/overview#backwards_compatibility>`_ 的特性支持的很好。对于 bitproto,直到 "
":ref:`v0.4.0 <version-0.4.0>` 这都是其一个主要的缺点。自从这个版本之后,bitproto "
"对于 bitproto,直到 "
":ref:`v0.4.0 <version-0.4.0>` 之前,向前兼容都是其一个主要的缺点。自从这个版本之后,bitproto "
"通过在编码后的字节流头部新增两个字节的方式实现了 :ref:`扩展性 <language-guide-extensibility>` "
"的支持,这两个字节存储了相关消息的占用字节数的多少。这个设计打破了 bitproto "
"传统的编码结构的设计,因为新增了一些对于编码大小的反射信息,因此这个功能被设计为可选的。"

#: ../../index.rst:145 299f1275a54e46b494de3d92413a4e4b
#: ../../index.rst:143 299f1275a54e46b494de3d92413a4e4b
msgid "Shortcomes"
msgstr "缺点"

#: ../../index.rst:147 9bebbcfbdbe34230bbc816acbb02c511
#: ../../index.rst:145 9bebbcfbdbe34230bbc816acbb02c511
msgid "Known shortcomes of bitproto:"
msgstr "已知的 bitproto 的缺点如下:"

#: ../../index.rst:149 4a6419cd37ae4ad3be9b5ee0652112c4
#: ../../index.rst:147 4a6419cd37ae4ad3be9b5ee0652112c4
msgid ""
"bitproto doesn't support varying sized types. For example, a ``unit37`` "
"always occupies 37 bits even you assign it a small value like ``1``."
msgstr ""
"bitproto 不支持变长数据类型。 举例来说,一个 ``uint37`` 类型的数据永远在编码后占用 37 个比特,即使你赋值它一个诸如 "
"``1`` 这样很小的值。"

#: ../../index.rst:152 8d2a5e4025d541ca939419923cce1100
#: ../../index.rst:150 8d2a5e4025d541ca939419923cce1100
msgid ""
"Which means there will be lots of zero bytes if the meaningful data "
"occupies little on this type. For instance, there will be ``n-1`` bytes "
Expand All @@ -259,7 +257,7 @@ msgstr ""
"这意味着,如果有意义的数据占用类型的占比较小的时候,编码后的数据中会有很多零字节。比如,当一个大小为 ``n`` "
"个字节的类型只被使用了其中一个字节的时候,会有 ``n-1`` 个字节会是零字节。"

#: ../../index.rst:156 b014a17d871142d68f138748e2e1d6dd
#: ../../index.rst:154 b014a17d871142d68f138748e2e1d6dd
msgid ""
"Generally, we actually don't care much about this, since there are not so"
" many bytes in communication with embedded devices. The protocol itself "
Expand All @@ -270,7 +268,7 @@ msgstr ""
"一般来说,我们并不特别关心这个问题,因为在嵌入式设备上的通信量一般不会有多少字节。这里的协议自身一般要设计地紧凑一些。如果你真的比较关心这个问题,可以考虑使用"
" `zlib <https://zlib.net/>`_ 类似的压缩机制,在消息编码后进行一次协议压缩。"

#: ../../index.rst:161 687204ff4d4e4b1bb0ade357e24192cb
#: ../../index.rst:159 687204ff4d4e4b1bb0ade357e24192cb
msgid ""
"bitproto can't provide :ref:`best encoding performance <performance-"
"optimization-mode>` with :ref:`extensibility <language-guide-"
Expand All @@ -279,7 +277,7 @@ msgstr ""
"bitproto 无法同时提供 :ref:`最佳的性能 <performance-optimization-mode>` 和 "
":ref:`扩展性能力 <language-guide-extensibility>`。"

#: ../../index.rst:164 e4026a353d514c908bec9320a3799370
#: ../../index.rst:162 e4026a353d514c908bec9320a3799370
msgid ""
"There's an :ref:`optimization mode <performance-optimization-mode>` "
"designed in bitproto to generate plain encoding/decoding statements "
Expand All @@ -293,7 +291,7 @@ msgstr ""
"中所有的类型都是定长的,因此我们可以在代码生成阶段就清楚地知道如何对其进行编解码。压测的结果表明,这个模式带来了巨大的性能提升,不过目前为止我还没有想到一个办法来支持"
" bitproto 的扩展性功能和这个性能优化模式一起工作。"

#: ../../index.rst:171 f6eed4ecd7f54ebb82d853f61b9a2132
#: ../../index.rst:169 f6eed4ecd7f54ebb82d853f61b9a2132
msgid "Content list"
msgstr "内容列表"

23 changes: 14 additions & 9 deletions docs/locales/zh/LC_MESSAGES/language.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ msgid ""
msgstr ""
"Project-Id-Version: bitproto 0.4.0\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-12-26 21:32+0800\n"
"POT-Creation-Date: 2023-05-22 21:23+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.11.0\n"
"Generated-By: Babel 2.12.1\n"

#: ../../language.rst:4 1f779ce1fb214cc7a376e7eb752bae1d
msgid "Language Guide"
Expand Down Expand Up @@ -401,8 +401,9 @@ msgstr "扩展性"
#: ../../language.rst:439 5f70af344e4f486d9190b6bd0e313be6
msgid ""
"Bitproto knows exactly how many bits a message will occupy at compile "
"time, because all types are fix-sized. This may make backwards-"
"compatibility hard."
"time, because all types are fix-sized. This may make `forward-"
"compatibility <https://en.wikipedia.org/wiki/Forward_compatibility>`_ "
"hard."
msgstr ""
"因为 bitproto 中所有类型都是定长的,因此 "
"bitproto编译器在代码生成阶段就可以清楚地知道一个消息会在编码后占用多少比特。但是,这给协议的兼容性设计造成了一点麻烦。"
Expand All @@ -411,11 +412,11 @@ msgstr ""
msgid ""
"It seems ok to add new fields to the end of a message in use, because the"
" structures of existing fields are unchanged, the decoding end won't scan"
" the encoded bytes of new fields, then \"the backward-compatibility "
" the encoded bytes of new fields, then \"the forward-compatibility "
"achieved\":"
msgstr ""
"如果我们把新增字段追加到消息的尾巴上,看上去似乎满足了协议的兼容性设计。因为已经存在的老的字段的结构是不变的,解码的一端不会扫描到新增的字段的数据,这样就实现了"
" \"协议向后兼容\" 。"
" \"协议向前兼容\" 。"

#: ../../language.rst:454 404d3fdb75a5452191b2846bf3735601
msgid ""
Expand Down Expand Up @@ -665,9 +666,11 @@ msgstr ""
#: ../../language.rst:662 d8243880ab7145a0ad9b686f35e3dbc1
msgid ""
"Syntax highlighting settings for PyCharm is available from `bitproto's "
"github repository <https://github.com/hit9/bitproto/tree/master/editors/pycharm>`_."
"github repository "
"<https://github.com/hit9/bitproto/tree/master/editors/pycharm>`_."
msgstr ""
"bitproto 的 PyCharm 语法高亮插件: `<https://github.com/hit9/bitproto/tree/master/editors/pycharm>`_."
"bitproto 的 PyCharm 语法高亮插件: "
"`<https://github.com/hit9/bitproto/tree/master/editors/pycharm>`_."

#: ../../language.rst:666 867939080c3643a6ad15e3ce30b70e9f
msgid "VSCode"
Expand All @@ -677,7 +680,9 @@ msgstr ""
msgid ""
"Install the extension from marketplace: `bitproto vscode extension "
"<https://marketplace.visualstudio.com/items?itemName=hit9.bitproto>`_."
msgstr "VSCode 插件可以直接从市场安装: `<https://marketplace.visualstudio.com/items?itemName=hit9.bitproto>`_."
msgstr ""
"VSCode 插件可以直接从市场安装: "
"`<https://marketplace.visualstudio.com/items?itemName=hit9.bitproto>`_."

#~ msgid "``int{8,16,32,64}``"
#~ msgstr ""
Expand Down
13 changes: 7 additions & 6 deletions docs/locales/zh/LC_MESSAGES/performance.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ msgid ""
msgstr ""
"Project-Id-Version: bitproto 0.4.0\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2023-01-02 22:16+0800\n"
"POT-Creation-Date: 2023-05-22 21:23+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.11.0\n"
"Generated-By: Babel 2.12.1\n"

#: ../../performance.rst:4 bb38bc42f11b4de68c861ba21eb53e25
msgid "Performance"
Expand Down Expand Up @@ -82,8 +82,9 @@ msgstr ""
"我在一个 `stm32 开发板 <https://www.st.com/content/st_com/en/products"
"/microcontrollers-microprocessors/stm32-32-bit-arm-cortex-mcus/stm32"
"-mainstream-mcus/stm32f1-series/stm32f103/stm32f103ze.html>`_ (arm "
"cortex-m3 72MHz cpu) 上, 对 ``100`` 字节大小的消息进行了压测测试, 一次编码调用或者解码调用占用 ``120~140 μs`` 左右的时间,并且可以通过 "
":ref:`优化模式 <performance-optimization-mode>` 来降低到 ``15 μs`` 左右。"
"cortex-m3 72MHz cpu) 上, 对 ``100`` 字节大小的消息进行了压测测试, 一次编码调用或者解码调用占用 "
"``120~140 μs`` 左右的时间,并且可以通过 :ref:`优化模式 <performance-optimization-mode>` "
"来降低到 ``15 μs`` 左右。"

#: ../../performance.rst:34 cc212dc5e38441c8ac7fe153e792ebcd
msgid ""
Expand Down Expand Up @@ -182,8 +183,8 @@ msgstr "性能敏感的场景,``100μs`` 和 ``10μs`` 完全不是一回事
#: ../../performance.rst:90 6e43e0071f2446cdb2e9b84f507785ea
msgid ""
"The firmwares of communication ends are always upgraded together, thus "
"the backward-compatibility is not so important."
msgstr "通信相关方的固件升级总是一起进行的,这样向后兼容性就不那么重要了。"
"the forward-compatibility is not so important."
msgstr "通信相关方的固件升级总是一起进行的,这样向前兼容性就不那么重要了。"

#: ../../performance.rst:91 57d569b743564fdab21876c78629b899
msgid "Firmware updates are not frequent, even only once for a long time."
Expand Down
2 changes: 1 addition & 1 deletion docs/performance.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ Optimization mode is designed for performance-sensitive scenarios, such as low p
compute-intensive microcontrollers. I recommend to use the optimization mode when:

* Performance-sensitive scenarios, where ``100μs`` means totally different with ``10μs``.
* The firmwares of communication ends are always upgraded together, thus the backward-compatibility is not so important.
* The firmwares of communication ends are always upgraded together, thus the forward-compatibility is not so important.
* Firmware updates are not frequent, even only once for a long time.

Specially, for the scenario that firmware-upgrading of communication ends have to be processed partially,
Expand Down

0 comments on commit 5bf13d5

Please sign in to comment.