ArkScript v4 pre-release 0.1

[4.0.0-0.1] - 2022-11-28

Added

• more tests for the io builtins
• added lines and code coloration in the error context
• new dependency: fmtlib

Changed

• instructions are on 4 bytes: 1 byte for the instruction, 1 byte of padding, 2 bytes for an immediate argument
• enhanced the bytecode reader and its command line interface
• added the padding/instruction/argumentation values when displaying instructions in the bytecode reader
• fixed underline bug in the error context
• the str:format functions now expects strings following this syntax: https://fmt.dev/latest/syntax.html
• more documentation about the compiler implementation
• more documentation about the virtual machine
• closures can be now be compared field per field: (= closure1 closure2) will work only if they have the same fields (name) and if the values match

Removed

• removed unused NodeType::Closure
• removing the custom string, replacing it with std::string (the format engine of the custom string had a lot of memory leaks)
• removing the custom string, replacing it with std::string (the format engine of the custom string had a lot of memory leaks)
• Utils::digPlaces and Utils::decPlaces got removed as they were no longer needed
• removed deprecated code (list:removeAt, ark executable now replaced by arkscript)

ArkScript v3.4.0

Added

• added new async and await builtins
  • they can access the outer scope
• added methods to create and destroy an execution context and a future in the VM
• added new CLI option --ast to generate JSON from the generated abstract syntax tree
• added an AST to JSON compiler
• added warnings on unused functions/quotes and statements without any effect

Changed

• printing a closure will now print its fields instead of Closure<1432>
• macros are always evaluated, even when they aren't any user defined macro
• argcount works on symbols and anonymous functions

Deprecated

• deprecating Value VM::resolve(const Value* val, Args&&... args)

Removed

• removed the std::ostream& operator<< of the Value, now using the .toString(stream, vm reference)
• removed the global VM lock
• removed coz and ARK_PROFILER

ArkScript v3.3.0

Added

• running the modules tests in the CI
• new bytecode instruction POP, removing the last value from the stack
• more documentation about ArkScript and its modules
• more tests for the io builtins
• added lines and code coloration in the error context
• added documentation about the compiler implementation
• added documentation about the virtual machine
• ArkScript now supports LTO if the compiler can do it
  • this is disabled in GCC 8 as this causes a runtime crash due to an ABI breakage

Changed

• fixed underline bug in the error context
• changing the way we count received arguments in arity / type errors for failed function call
• the CLI can now take a list of paths to the standard library, separated by ';'

ArkScript v3.1.3

Added

• adding an ExecutionContext to host the pointers (instruction, page, stack) and execution related structures (stack, locals, scopes), to ease the transition to a parallelized VM
  • the VM can have multiple independant context running on the same bytecode
• the VM now takes a reference to an Ark::State instead of a raw non-owning pointer
• adding ARK_PROFILER_MIPS to toggle instruction per second calculation
• adding new way to typecheck in builtins
• new CI build step now running valgrind to check for memory leaks
• new type checker (to be used by builtins)
• better type errors generation (with the list of arguments, if they are matching or not, and more)

Changed

• splitting Utils.hpp into multiple files for easier maintenance and contextualisation
• reserving a default scope size of 3, which yields really good performance results compared to nothing being reserved
• upgrading the builtins error handling to use the BetterTypeError
• the VM now displays the debug info (ip, pp, sp) at the end of the backtrace instead of the beginning

Removed

• BetterTypeError has been removed in favor of a type checker using templates and an error generator

Deprecated

• deprecating VM(State*) in favor of VM(State&)

ArkScript v3.1.2

Added

• adding support for append_in_place, concat_in_place, pop_list and pop_list_in_place in the bytecode reader
• added page_ptr(int) in the compiler to replace &page(int)
• added literals _u8 and _u16
• added table overflow detection in the compiler, to avoid creating unusable bytecode (checks if the symbols/values table is full or not)
• new Installer.iss (inno setup script) to generate a Windows installer
• new exceptions for type errors

Changed

• using doc_formatting.first_column instead of doc_formatting.start_column when displaying the CLI help
• brand new cmake build system
• renaming Ark/Config.hpp to Ark/Platform.hpp
• refactored compiler handling of keywords
• removed using Inst_t = uint8_t in the compiler
• moved everything related to the AST in Ark/Compiler/AST/
• moved everything related to the macros in Ark/Compiler/Macros/
• renamed unclear file CValue to ValTableElem
• the parser is now an internal class
• the AST Optimizer was moved to Compiler/AST
• changed the ARKSCRIPT_PATH to be a collection of paths to look into, separated by ;
• updating replxx to avoid a bug when compiling with clang

Removed

• removed ARK_SCOPE_DICHOTOMY flag so that scopes don't use dichotomic search but a linear one, since it proved to be faster on small sets of values. This goes toward prioritizing small functions, and code being cut in multiple smaller scopes
• removing download-arkscript.sh from the repo
• removed isFraction, isInteger, isFloat from Ark/Utils.hpp (worked on strings and used regex)
• removed mpark variant to use standard variant
• Ark::FeatureFunctionArityCheck was removed, making arity checks mandatory

ArkScript v3.1.1

Added

• ArkDoc documentation for the builtins
• Now using clang-format to ensure the code is correctly formatted

Changed

• the macro processor can now handle multiple macro definitions in a if-macro: !{if true { !{a 1} !{b 2} }} is finally working

Deprecated

• ark command is now marked as deprecated, in favor of arkscript

ArkScript v3.1.0

Change Log

[3.1.0] - 2021-06-29

Added

• adding of new string function for manipulation of utf8 string (str:ord and str:chr)
• utf8 support for lexer
• UserType::del(), used only by the virtual machine to free memory
• a new unique stack based on a std::array<Value, ARK_STACK_SIZE>, the default stack size being 8192
• more profiling tests
• more options on the display() method of the bytecode reader, allowing us to selecto segment of bytecode instead of displaying everything
• added a new token type: Spread ; it handles ...identifier, needed in macros
• the parser can now handle macros definitions
• macros are being handled right after the parsing, before the AST optimizer can run
  • if macros: !{if compile-time-value then [optional else]}
  • values macros: !{name value}
  • functions macros: !{name (a b c ...args) body}
• sys:platform, containing the current platform name
• updated the CLI so that we can slice the bytecode when displaying it
• the bytecode reader can now display
  • all the segments
  • only the values segment
  • only the symbols segment
  • only the code segment (all of them or a given one)
  • only the segments' titles and length
• verifying that we give enough arguments
• we can now import macros from other files
• undefining macros is now possible by using !{undef macro_name}
• str:join added in the standard library
• str:split can now take longer separators
• added symcat in macros to concatenate a symbol and a number/string/symbol to create a new one
• added argcount in macros to count (at compile time) the number of arguments of a function
• fixed a bug where (bloc) and (print bloc), given a !{bloc value} macro, didn't give the same result (one was applied, the other was partial)
• new module to manipulate bits: bitwise
• enhanced standard library

Changed

• updating doxyfile and some docstrings
• updating the download script
• enhancing examples
• creating a Scope allocates 4 pairs instead of 2, reducing the number of reallocations needed
• tailOf renamed to tail and headOf to head ; no need to keep the relics of the past
• headOf (now head) returns the real head of a container (List or String), the first element (nil if the list is empty, "" if the string is empty)
• the http module was updated to add http:params:toList and fix the http:server:[method] when passing a function
• fixing the compiler when we encounter get fields in lists
• updating the parser to support usually invalid constructions when they are in macros, to allow things like !{defun (name args body) (let name (fun args body))}
• updated the lexer to add UTF8 support and allow unconventional identifiers as long as they aren't keyword nor operators, so things like -> now works
• fixing the code optimizer to avoid removing unused variables which are defined on function calls
• fixed the traceback generation on errors, it should now display the correct function names
• reorganizing the compiler code
• reorganizing the parser code to make it more maintainable
• adding make_node<T> and make_node_list internally to avoid repetitive code
• enhancing the parser atom method
• enhancing the way we choose the subparser to use in the parser
• avoid using std::endl if it's not useful
• CI was split into multiple files to ease maintenance
• moving ArkScript tests from tests/*.ark to tests/arkscript/*.ark
• fixed macros adding useless begin blocks, sometimes breaking code generation from macros
• moving std lib related tests into std/tests/
• fixed macro chaining
• fixed lexer, which wasn't adding the last token it read under some specific conditions

Removed

• ~UserType, since we are doing manual memory management now
• Frame were removed because they were giving bad performances
• firstOf was removed because it's basically a (@ list 0) and it was doing the job of head
• Ark::Utils::toString, our internal version of std::to_string
• use of static in the MacroProcessor and in the NodeType to string conversion function
• Ark::Logger was removed in favor of std::cout/cerr + termcolor

ArkScript v3.0.15

3.0.15

Added

• new submodule, plasma-umass/coz (a profiler)
• macros for profiling, enabled only if ARK_PROFILE is defined
• cmake flags using -D to turn on/off sys:exec and the coz profiler
• mpark variant is now the default used instead of the default STL variant (faster, better, stronger, and its creator is quite a god)
• new cmake flag, -DARK_SCOPE_DICHOTOMY=On|Off (default Off)
• using internal only references to constants and symbols to reduce the number of useless copies of the value type

Changed

• updated standard library
• updated modules, adding hash
• updated the error handlers to avoid errors (sigsev) when handling errors (lexing, parsing, optimization and compilation error)
• better error message at runtime when a plugin can not be found
• fixes issue #203 (imports are ill-formed when given an absolute path)
• fixes issue #205 (search for the standard library folder in more common places)
• transitioning from C++ streams to printf
• replaced the thirdparty/ folder with a git submodule in thirdparties/
• now checking that a scope doesn't have our symbol before doing a mut operation (in dichotomic mode it was automatically handled, but not in linear mode)
• enhancing the cmake defines (-DARK_XYZ) and the code using them
• lighter Frame (from 40B to 32B), moved some unrelated logic from the frame to the virtual machine
• (sys:exec) now returns the stdout output of the given command

ArkScript v3.0.14

Changelog

3.0.14

Added

• the parser can handle (let|mut a b.c) (bug fix)
• f[ruv|no-ruv] CLI switch to control the optimizer (ruv stands for remove unused variables)
• error message when we have too many parenthesis (at parse time)
• error message when using an operator not right after a (
• error message when we're capturing an unbound variable
• added (sys:exit code) as a builtin
• bytecode integrity checking through a sha256 in the header
• tests for math:fibo and math:divs
• added the ability to give scripts arguments, through sys:args

Changed

• the parser checks if set is given a dot expression as an identifier (which is an error)
• the parser should take in account captured variables as well, otherwise some variables are optimized while they are captured, resulting in runtime errors
• better unbound variable error message
• (implementation) every constructor with a single argument is now marked as explicit
• REPL does not need to add extra surrounding {}
• the Ark::State (re)compiles a file even if there is a bytecode version available
• the parser is now stricter and gives better error messages when we give too many/not enough arguments to a keyword
• better handling of the code given to the REPL (adds new line)
• renamed the executable from Ark to ark
• now using Github Actions instead of Travis
• the parser can now detect when let/mut/set are fed too many arguments, and generate an error
• the compilater now handles (set a b.c.d)
• using a new plugin interface, more C-like

Removed

• class Ark::internal::Inst which was used as a wrapper between uint8_t and Instruction
• worthless examples were removed
• removing f[no-]aitap since it wasn't used anymore in the code

ArkScript v3.0.13

3.0.13

Added

• string tests
• list tests
• range tests
• unbound variable checker at compile time (won't break on plugin symbols)

Changed

• list:find returns -1 to stay consistent with str:find
• hot fix (mut a 10) (let b 12) (set a b) (set a 11), the immutability was transfered from b to a
• converting list, append and concat to instructions
• instructions LIST CONCAT and APPEND added to replace the corresponding builtins