Sean/purus

.envrc

@@ -0,0 +1 @@
+use flake

.gitignore

@@ -19,14 +19,15 @@ bower_components/
 node_modules
 tmp/
 .stack-work/
-output
+# output
 tests/purs/docs/docs/
 core-tests/full-core-docs.md
 tests/support/package-lock.json
 .psc-ide-port
 .psc-package/
 tags
 TAGS
+.nvimrc
 
 # Gather source map files from golden tests
 .source-maps
@@ -38,3 +39,8 @@ TAGS
 *.ps
 *.svg
 tests/purs/make/
+.direnv/
+/.pre-commit-config.yaml
+/result*
+
+tests/purus/passing/**/cache-db.json

Architecture.md

@@ -0,0 +1,100 @@
+# Architecture
+
+This document contains an overview of the *current* architecture of our compiler pipeline. It is not meant to be exhaustive and is intended to serve as a guide for onboarding new developers / make the giant PRs more accessible. 
+
+
+## Step 1: Type Deduction/Synthesis (Language.PureScript.CoreFn.Desugar)
+
+During the research phase of this project, we determined that PIR (Plutus Intermediate Representation) should be our ultimate compilation target (with final compilation to UPLC handled by the PIR compiler). Because PIR is an explicitly typed language, and because the vanilla `CoreFn` AST is not explicitly typed, it is necessary to convert the `Language.PureScript.AST` AST into a typed variant. 
+
+Because conversion to `CoreFn` occurs after the initial (vanilla PS) typechecker pass, we receive *most* expressions (but not all of them) annotated with the inferred or explicitly declared type. Desugared expressions or declarations involving type classes, however, are not ignored by the PS typechecker, but we require explicit annotations for explicit type class dictionaries and the functions that operate on them. 
+
+This step consists of two main phases: 
+
+  - First, we traverse every *type* annotation in the vanilla PS `Module` that is the input to our conversion function and desugar constraint types to types that require explicit dictionary arguments. E.g. `Eq a => a -> (...)` becomes `Eq$Dict a -> a -> (...)`. 
+  - Next, we deduce the types. Because top-level declarations are always explicitly typed, we proceed "top-down" from the top-level signature as far as we can, and switch to a "bottom-up" synthesis strategy in cases where the type cannot be determined by the top-level signature. 
+
+Some notes: 
+
+  - We want to preserve quantifiers in this step. While some quantifiers can be eliminated during monomorphization, there is no guarantee that *all* of them will be eliminated (this is not a problem, because PIR supports universal quantification). This requires special machinery when performing type deduction on expressions that contain quantified polymorphic functions, since we must take care to ensure that the type variables "line up". 
+  - We run our desugarer in the same monad stack as the PS TypeChecker, but this is largely a convenience (since it lets us use the PS machinery for binding local variables or type variables, etc). We should *never* perform type inference or make any calls to `unify`. 
+  - The trickiest part of this is ensuring that type information for explicit type class dictionaries is introduced at the correct point. Again, type class dictionaries are not processed by the PS typechecker. 
+
+## Step 2: Monomorphization & Inlining (Language.PureScript.CoreFn.Convert.Monomorphize)
+
+PureScript's implementation of Records employs Row Types. Moreover, PureScript supports *polymorphic* records, which are backed by *open rows* (e.g. `{a :: Foo | r}`). 
+
+Records in PureScript (unlike in Haskell) are not just syntatic sugar for products - the order of fields in the record is *not* determined by the order of fields in the declaration or expression that introduces the record. *An* order can be established for any fully instantiated (i.e. closed - doesn't contain any type variables of kind `Row Type`) record types - we choose a lexicographic ordering to mirror PureScript's `RowToList` sorting but this is not essenital. 
+
+We must, therefore, perform monomorphization on record types in order to transform Object Literals to literal products, transform record accessors into `case` analysis of the product that corresponds to the original record, and transform record updates into product updates. 
+
+Because a single polymorphic record type may be instantiated to different concrete types at different places in the AST, we must also inline while we monomorphize. 
+
+The general monomorphization procedure is as follows (not a full algorithm): 
+
+  1. We traverse the AST until we reach an `App` node. 
+  2. We "peel" the App node and get an `(f,args)` where `f` is the "terminal" function expression and `args` is a list of the argument expressions to which it is applied. 
+  3. We check whether the type of `f` is already monomorphic. If it is, we make no change to the node. 
+  4. If `f` is not monomorphic, we strip the quantifiers and check whether we can specialize any bound type variables to a concrete type. E.g. if `f :: forall x. Tuple x x -> x -> Int` and the arguments are `[Tuple String String, String]`, we instantiate `x` to `String`. 
+  5. We recurse down the list of arguments. If we encounter an argument that is a free (term-level) variable that is *not* a Plutus builtin, we inline it, monomorphizing it to the concrete type of the argument if possible.  
+  6. We apply the (possibly monomorphized & inlined) argument to the function and procede to the next argument 
+     until we run out of arguments to process.
+  7. Finally, we re-quantify the type of `f` in the resulting expression. (It may contain free type variables of kind `Type`, which can be represented in PIR)
+
+The procedure for inlining is: 
+  1. We check whether the (term level) variable we aim to inline is locally scoped (i.e. qualified by source position) or globally scoped (e.g. qualified by module name). For now, we only attempt to inline globally scoped variables (...I forget why...)
+  2. If the variable is globally scoped (i.e. is the name of a module-level declaration), we lookup the body of the declaration. There are two possibilities here: Either the declaration will be a single non-recursive binding, or it will be a member of a mutually recursive binding group. (PS compiler sorts these for us) 
+    - If the declaration is non-recursive we "walk" its expression-body and monomorphize/inline the sub-expressions as necessary in order to properly assign the expression the type that it is to be monomorphized to. 
+    - If the declaration the member of a recursive binding group, we pause inlining, walk the expression, and "collect" a `Map Name (Name,SourceType,Expr)` where the key is the original name of the expression, and the values are, respectively: The new "fresh" name to give to the monomorphized expression, the monomorphic type that we must assign the expression to, and the body of the declaration. We do this recursively until we have collected every member of the recursive binding group used in the target expression. Finally, we use that map to construct a *monomorphic* mutually recursive binding group (where the names are all fresh) and create a `Let`-binding for the monomorphized mutually recursive group. 
+
+The implementation of the monomorphizer/inliner consists in a few key functions that call each other recursively. To make reviewing easier, here's a brief explanation of what the key functions do (or are supposed to do at any rate): 
+
+  - `monomorphizeA` is the entry point that checks whether the node is an `App`, peels the arguments and function parts from the `App`, and calls `handleFunction` on the function expression and its arguments. 
+  - `handleFunction` branches depending on the function expression it is passed: 
+    - If it is passed a `Var` qualified by modulename, and the modulename is `Builtin`, it just returns the variable (since Builtins cannot be meaningfully inlined). 
+    - If it is passed an `Abs`, `handleFunction` tries to instantiate the type variables of the function type with corresponding concrete types of the arguments. If it succeeds, it subsitutes the concrete type in for the bound type variable in all sub-expressions and their types, then calls itself recursively on the body of the `Abs` until the type has been fully monomorphized. 
+    - If it is passed a `Var` that is not a builtin, `handleFunction` attempts to inline the expression the `Var` refers to by calling `inlineAs` with the monomorphized type. If this succeeds, `handleFunction` calls itself recursively with the monomorphized/inlined expression. 
+    - If it is passed anything else as the `f`, it checks whether the function type is monomorphic. 
+      - If the `f` is monomorphic, it applies it to its arguments and returns the resulting expresion. 
+      - If the `f` is not monomorphic, `handleFunction` throws an error. 
+  - `inlineAs` performs inlining and implements the above algorithm. Note that `inlineAs` is passed a PS `Type` argument, which represents the type that the expression corresponding to the `Name` being inlined *should* have after inlining. 
+  - `monomorphizeWithType` rewrites the type of an expression to match the supplied type and (much more importantly) rewrites the types of all sub-expressions to conform with the supplied type of the top-level expression. 
+
+## Step 3: Object desugaring and final IR (Language.PureScript.CoreFn.[IR / DesugarObjects])
+
+By the end of step 2, all polymorphic records have been monommorphized that can be monomorphized, but record-specific expression nodes (object updates/accessors/literals) still remain in the AST. In order to ensure that all "invalid" expressions and types have been desugared/eliminated prior to final PIR conversion, we define a restricted AST and `Type` type such that only expressions which can be converted into PIR can be represented - a kind of "parse-don't-validate" approach. (This AST is implemented with the `Bound` library.) 
+
+At this stage, we construct a set of dictionaries for type and data constructors. When constructing these maps, we add an arbitrary number of constructors for anonymous products (i.e. tuples) to accommodate objects. Specifically, we add constructors for tuples of up to 100 (they look like `data $GEN.~Tuple1 a = $GEN.~Tuple1 a` etc). These dictionaries serve two purposes: 
+  - We construct them in such a way that the SOP representation of the underlying data type is very explicit. I.e. for each *type* constructor, we construct an `[(Int,[Type])]` where the `Int` represents the corresponding data constructor's index in the data type, where this information (the constructor's index & arguments) is also available in the dictionary for each *data* constructor. (Due to the implementation of the PS AST, we need both of these dictionaries, even though in principle only the tycon dictionary is necessary)
+  - Embedding generated tuple types into these dictionaries allows us to treat desugared records as "normal" data types in the final compilation stage (i.e. they don't require any special handling). 
+
+Conversion into this restricted IR AST is, aside from object expressions, very straightforward. Therefore, in the rest of this section I will explain the object desugaring process. 
+
+### Object Literals 
+
+  1. We extract a `Map PSString SourceType` from the object literal expression by inspecting the expressions in each field. 
+  2. We sort the fields lexicographically (which establishes the uniform order of the product that will be constructed). 
+  3. We generate "fake" names for the product's type and constructor. E.g. `$GEN.~Tuple2 a b`. 
+  4. We construct a specialized function type for the product's data constructor using the sorted arguments. 
+  5. We assemble the literal with the "fake" constructor function and the arguments. 
+
+### Record Accessors 
+  1. We perform steps 1-4 of Object Literals conversion on the expression the accessor is being applied to, except we construct a Constructor `Binder` (a pattern), where the arguments are wildcard binders for every argument except the one that corresponds to the field being accessed. 
+  2. We use that pattern to construct a case expression that extracts the value corresponding to the field. 
+    - This is kind of hard to explain without an example. Suppose we have `foo = {a: 1, b :: "yup"}` in `foo.b`
+    - That gets turned into (something like) `case foo of {$GEN.Tuple2 _ $HERE -> $HERE} `
+
+### Record Updates 
+  1. Proceeds much like the accessor case, except we return a product in our case expression instead of returning a field. 
+    - Using the above definition of `foo`, if we have `foo {b = "hello"}`, this turns into: 
+    - `case foo of {$GEN.Tuple2 a _ -> $GEN.Tuple2 a "hello"}`
+
+## Step 4: Final Conversion to PIR (Language.PureScript.CoreFn.Convert.ToPIR)
+
+The final step of compilation is conceptually simple: We match on the constructors of our final IR and translate expressions (and the types they contain) into PIR `Term`s and `Type`s, using some machinery to generate fresh names when we need to construct a lambda (which should only happen when desugaring case expressions).
+
+NOTE: The *implementation* of case expressions is incredibly complex. At the moment we support matching on simple constructor patterns. Going forward, we ought to weak the final IR so that case expressions are presented in a form that makes it simpler to handle. 
+
+
+
+

CompilerBugs.txt

@@ -0,0 +1,135 @@
+NOT COMPILING:
+
+* mutuallyRecursiveBindinGroup
+  - Not abstracting variables correctly in lambda bodies
+  - Probably we use an abstract fn that only looks for the var bound in the lambda, should be easy to fix
+-------------
+letrec
+  !f_111 : integer -> integer = \(x_112 : integer) -> g_109 2
+  !g_109 : integer -> integer = \(y_110 : integer) -> h_106 (f_105 y_110) 3
+  !h_106 : integer -> integer -> integer
+    = \(x_107 : integer) (y_108 : integer) -> y_108
+in
+g_109 3
+*** Exception: user error (Error during PIR typechecking:
+Free variable at  () :  f)
+-------------
+
+
+* mutuallyRecursiveBindinGroupNoTypes
+  - Same as above
+
+
+* aFunction4
+  - This *should* fail b/c it mentions a polymorphic record in the type sig
+  - it works when applied to an argument that lets us monomorphize, see aFunction5
+
+
+* aFunction6
+  - Fails during object desugaring, seems like an unknown remains in there, likely related to the above list issue
+-------------
+let
+  go#185 = \(v#186 :: forall (z :: Prim.Type). ((z :: Prim.Type) -> (Prim.Int))) ->
+               (10 :: Prim.Int)
+in \(any#181 :: Array (t148)) ->
+       \(f#182 :: (forall (y :: Prim.Type). ((y :: Prim.Type) -> (Prim.Int)) -> (Prim.Int))) ->
+           f#182   # any#181
+     # Prim.Nil
+     # go#185
+*** Exception: user error (Unsupported type:
+  t148)
+-------------
+
+
+* recF1 / recG1 (mutually recursive)
+  - We're getting 'free variable' errors, probably what's happening is that
+    we need to bind all top level module identifiers like we do for let binding groups (should be ez fix)
+-------------
+/\x_96 -> \(x_48 : x_96) -> recF1_47 {x_96} x_48
+*** Exception: user error (Error during PIR typechecking:
+Free variable at  () :  recF1)
+-------------
+
+
+* nestedApplications
+  - Case expression wasn't eliminated properly. This is easy to fix, we just need to port over the
+    catchall handling machinery from `desugarConstructorPattern` to `desugarLiteralPattern`
+-------------
+*** Exception: Case expressions should be eliminated by now, but found:
+
+case v#86 of
+   2 ->
+     (3 :: Prim.Int)
+   _ ->
+     (5 :: Prim.Int)
+CallStack (from HasCallStack):
+  error, called at src/Language/PureScript/CoreFn/Convert/ToPIR.hs:269:17 in purescript-0.15.13-inplace:Language.PureScript.CoreFn.Convert.ToPIR
+-------------
+
+
+* consEmptyList1 / consEmptyList2
+  - Same error as the other list stuff
+-------------
+(/\a_191 ->
+   \(x_136 : integer) ->
+     /\a_191 -> \(xs_137 : Array_190 a_191) -> Cons_194 {integer} x_136 Nil_193)
+  1
+  Nil_193
+*** Exception: user error (Error during PIR typechecking:
+Type mismatch at () in term
+  '[ { Cons (con integer) } x ]'.
+Expected type
+  '(fun * *)',
+found type
+  '[ Array (con integer) ]')
+-------------
+
+
+* testEqViaOrd
+  - We expect the dictionary to have a record type in DesugarObjects but getting the dict via
+    superclass is a fn from the dict to the subclass method
+-------------
+\(dictOrd#129 :: forall (a :: Prim.Type). ((Lib.Ord$Dict ((a :: Prim.Type))) -> (((a :: Prim.Type) -> (((a :: Prim.Type) -> (Prim.Boolean))))))) ->
+    \(a#130 :: (a :: Prim.Type)) ->
+        \(b#131 :: (a :: Prim.Type)) ->
+            \(dict#125 :: forall (a :: Prim.Type). (Lib.Eq$Dict ((a :: Prim.Type)))) ->
+                case dict#125 of
+                   Eq$Dict v126@({ eq :: ((a :: Prim.Type) -> (((a :: Prim.Type) -> (Prim.Boolean)))) }) ->
+                     (v#126).eq
+                  # a#130
+              # (dictOrd#129).Eq0   # ({} :: (Prim.Record ({})))
+              # b#131
+*** Exception: ERROR: Record expression:
+  dictOrd#129
+  should have a Record type, but instead has type:
+  (Lib.Ord$Dict ((a :: Prim.Type)))
+CallStack (from HasCallStack):
+  error, called at src/Language/PureScript/CoreFn/Convert/DesugarObjects.hs:444:35 in purescript-0.15.13-inplace:Language.PureScript.CoreFn.Convert.DesugarObjects
+-------------
+
+
+* testValidatorApplied
+  - Free type variable. We lost the quantifiers somewhere here. Probably in the monomorphizer? This seems annoying b/c it worked last week, some of the other fixes must have broken it -_-
+  - Weirdly, the unapplied version works. I think that adding the explicit type abstractions is interacting weirdly with the monomorphizer.
+-------------
+==========END desugarCoreDecl: aBool==========
+\(datum#20 :: Prim.String) ->
+    \(redeemer#21 :: (b :: Prim.Type)) ->
+        \(context#22 :: (c :: Prim.Type)) ->
+            Prim.True
+  # ("datum" :: Prim.String)
+  # ("redeemer" :: Prim.String)
+  # ("context" :: Prim.String)
+tryConvertExpr result:
+\(datum#20 :: Prim.String) ->
+    \(redeemer#21 :: (b :: *)) ->
+        \(context#22 :: (c :: *)) ->
+            Prim.True
+  # ("datum" :: Prim.String)
+  # ("redeemer" :: Prim.String)
+  # ("context" :: Prim.String)
+
+*** Exception: Free type variable in IR: c
+CallStack (from HasCallStack):
+  error, called at src/Language/PureScript/CoreFn/Convert/Datatypes.hs:186:16 in purescript-0.15.13-inplace:Language.PureScript.CoreFn.Convert.Datatypes
+-------------

cabal.project

@@ -1,2 +1,21 @@
+repository cardano-haskell-packages
+  url: https://input-output-hk.github.io/cardano-haskell-packages
+  secure: True
+  root-keys:
+    3e0cce471cf09815f930210f7827266fd09045445d65923e6d0238a6cd15126f
+    443abb7fb497a134c343faf52f0b659bd7999bc06b7f63fa76dc99d631f9bea1
+    a86a1f6ce86c449c46666bda44268677abf29b5b2d2eb5ec7af903ec2f117a82
+    bcec67e8e99cabfa7764d75ad9b158d72bfacf70ca1d0ec8bc6b4406d1bf8413
+    c00aae8461a256275598500ea0e187588c35a5d5d7454fb57eac18d9edb86a56
+    d4a35cd3121aa00d18544bb0ac01c3e1691d618f462c46129271bccf39f7e8ee
+
+
 packages:
   purescript.cabal
+
+-- HACK: plutus core cannot build without it, remove after bump.
+constraints:
+  nothunks < 0.2
+
+index-state:
+  , cardano-haskell-packages 2024-06-27T10:04:00Z