Skip to content

lessons

Jump to bottom
Miguel Gamboa edited this page May 29, 2024 · 17 revisions

Lessons:

Lesson 01 - Introduction and Modern VMs, e.g. JVM, Node

  • Bibliography and Lecturing methodology: github and slack.
  • Tools: javac, javap, kotlinc, JDK 17 and gradle.
  • Program outline in 3 parts:
    1. Java Type System and Reflection;
    2. Metaprogramming and Performance;
    3. Lazy processing.
  • Project in 3 parts according to program outline.
  • Managed Runtime or Execution Environment or informally virtual machine (VM) or runtime.
  • Execution Environment includes:
    • Compiler,
    • Programming languages,
    • Standard libraries,
    • Dependency manager (e.g. gradle, maven)
    • Central Repository (e.g. Maven Central Repository, Nuget, NPM, etc)
  • Examples of languages targeting JVM: Java, kotlin, Scala, Clojure.
  • Examples of languages targeting Node: JavaScript, TypeScript, Kotlin.
  • JVM runs .class files with bytecode
  • JVM translates bytecode to machine code (e.g. IA-32, AMD64, etc depending of the CPU)
  • In Javascript ecosystem modules are deployed in source code i.e. Javascript.
  • Distinguish between Programming Language <versus> VM

Lesson 02 - Metadata, Classpath and Java Type System introduction

  • One file .class for each class definition
  • Metadata:
    • data that provides information about other data
    • In a .class the metadata provides a description and structure of a Type.
  • Using javap -c -p AppKt.class to inspect metadata and bytecode definition of class AppKt
  • CLASSPATH
    • e.g. -cp . - local folder
    • e.g. -cp '.:~/JetBrains/IntelliJIdea2022.2/plugins/Kotlin/lib/*'
    • (for windows use ; rather than :)
  • Type System - Set of rules and principles that specify how types are defined and behave.
  • Two kinds of types: Primitive and Reference types.
  • Classes have Members
  • Members may be: Fields or Methods.
  • There are NO properties in Java Type System.
  • Using javap -p StudentPerson.class to inspect metadata
  • The fully qualified name of a class includes its package name.
  • Constructor is a method with the name <init> returning void.

Lesson 03 - Members and static vs instance (non-static)

  • Member access syntax: Receiver.Member.
  • The Receiver is the target of the member access and it is a Type (for static members) or an Object (for non-static/instance members).
  • JOL - Java Object Layout:
    • java -cp .;jol-cli-0.17-full.jar org.openjdk.jol.Main estimates <classqualifiedname>
    • (linux replace ; by : on -cp (classpath))
  • Object header = mark word (used for hash, locks, GC, etc) + class word (class-specific metadata).
  • Fields alignment, reorder and padding gap.
  • Immutable values:
    • constant value calculated at compile time, e.g. Kotlin const
    • immutable, yet dynamically initializable

Lesson 04 - Boxing and Unboxing

  • Static initializer: initializes static fields.
  • Boxing and Unboxing.
  • Nested classes.
  • Inner classes: non-static in Java or inner in Kotlin.
  • Abstract classes cannot be directly instantiated.
  • Interfaces represent abstract types that cannot be instantiated too.
  • Override
  • Names collision and Member access ambiguity
  • Methods call resolution
  • Anonymous classes.

Lesson 05 - Nested, Inner, Abstract classes, Virtual Methods

  • Nested classes.
  • Inner classes: non-static in Java or inner in Kotlin.
  • Abstract classes cannot be directly instantiated.
  • Interfaces represent abstract types that cannot be instantiated too.
  • Override
  • Names collision and Member access ambiguity
  • Methods call resolution: Fields, Methods, and Virtual Methods.

Lesson 06 - Anonymous Types

  • Anonymous Classes in Java
  • Object Expressions in Kotlin
  • Resulting types from compilation:
    • Java: always includes a field this$0 to the outer class (i.e. inner)
    • Kotlin: only includes a field this$0 if it uses the outer class scope.

Lesson 07 - Kotlin to Java type system

  • Homework 2 - virtual and non-virtual methods.
  • Kotlin Class Members
  • Analyzing Kotin properties in JVM.
  • There are NO properties in Java Type System.
  • A Kotlin property may generate:
    • Backing field
    • Getter, i.e. function get... -- called with invoke... bytecode.
    • Setter, i.e. function set... (if defined with var).
  • Top-level declarations
  • Extension functions

Lesson 08 - Object declarations

  • Singleton design pattern
  • object keyword:
    • private constructor
    • Singleton instance in static INSTANCE field;
  • companion object - specific type of object declaration associated with its owner class.
  • Function Types
  • Kotlin compiler generates an _anonymous class: that implements the interface aligned with the respective function type.

Lesson 09 - Kotlin Reflection

  • Reflection object oriented API for metadata
  • Reflection ---> metadata ---> Type System
  • Type System: types have members
  • Kotlin Reflection API: KClass ----->* KCallable
    • An instance of KClass may represent a type in Kotlin.
    • An instance of KCallable may represent a member in Kotlin.
  • KCallable base type of KFunction and KProperty
  • KProperty and KMutableProperty
  • KCallable ----->* KParameter
  • KFunction properties: name, type, parameters and instanceParameter.
  • KParameter property kind: INSTANCE versus EXTENSION_RECEIVER
  • KParameter property isOptional
  • KClass::createInstance()
  • KFunction::call()

Lesson 10 - KClass, KType and classifier

  • To directly reference a KType, we use the typeOf function:
    • e.g. func.returnType != typeOf<Unit>()
  • KType holds information about nullability and type arguments.
  • KType properties: isMarkedNullable, arguments, and classifier.
    • arguments provide information about the type arguments (i.e. List<KType>)
    • classifier provides a reference to the associated class (i.e. KClassifier).
      • KClassifier is the base type of KClass
  • Implement an utility extension Appendable.log(obj:Any)
  • isAccessible - Provides a way to suppress JVM access checks for a callable.
  • Test with Kotlin domain classes and Java domain classes
  • Appendable.logGetters(obj:Any) in Kotlin to inspect Java getters:
    • methods with prefix get
    • a single argument corresponding to the instance parameter
    • return type different from Unit:
    • e.g. m.returnType.classifier != Unit::class

Lesson 11 - NaiveMapper

  • NaiveMapper, takes inspiration from libraries like AutoMapper or MapStruct:
    • Simplify the process of mapping data between objects of different types by copying values from properties of one object to corresponding properties of another object.
    • Offers an extension function like Any.mapTo(dest: KClass<*>): Any
  • 1st version - through mutable properties (i.e. KMutableProperty):
    • The destination type must have a parameterless constructor
    • The source and destination properties share the same name and type
    • The destination properties are mutable
  • 2nd version - through constructor parameters:
    • Call constructor via: fun call(vararg args: Any?): R
  • 3rd version - avoid Reflect on mapping function:
    • NaiveMapper<T : Any>(val srcKlass: KClass<*>, val destKlass: KClass<T>):
      • Look for matching properties once in initalization
    • fun mapFrom(source: Any): T - instantiate destKlass with values from source

Lesson 12 - Annotations

  • Annotations in the JVM are a form of metadata that can be added to Java classes, methods, fields, and other program elements.
  • Annotations are strongly typed
  • Each annotation inherits from java.lang.annotation.Annotation
  • E.g. JUnit annotation @Test corresponds to the following type:
public interface org.junit.Test extends java.lang.annotation.Annotation{...}
  • Kotlin Reflect API on annotations:
    • annotations: List<Annotation>
    • findAnnotation<T>(): T
    • hasAnnotation<T>(): Bool
  • When a Kotlin member generates multiple Java members, there are multiple potential locations.
  • Use site target to explicitly specify the destination location within the metadata:
    • e.g. @property:MapProp("from") val country: String
  • Specify the allowed elements with the @Target annotation.
  • Enhance NaiveMapper to map properties to parameters with different name through the annotation @MapProp

Lesson 15 - Reflect Generics

  • NaiveMapperrecursive mapping complex type properties with auxiliary NaiveMapper instances.
  • Mapping instances of List
  • E.g. from a property of type List<Song>
    • prop.returnType.classifier is List::class
    • prop.returnType.arguments[0].type is typeOf<Song>
    • prop.returnType.arguments[0].type.classifier is Song::class

Lesson 18 - Reflection Exercises

  • fun KClass<*>.isSubclassOf(base: KClass<*>): Boolean
  • fun KClass<*>.isSuperclassOf(derived: KClass<*>): Boolean
  1. You have to develop a ComparerOrder<T : Any>(klass: KClass<T>) that implements Comparator and it is able to compare instances of type represented by klass, according to the properties which are both: Comparable and annotated with Comparison. Notice that you should compare respecting the order specified in annotation. Example:
class Student (
  @Comparison(2) val nr:Int,
  val name: String,
  @Comparison(1) val nationality: String,
)
 
val s1 = Student(12000, "Ana", "pt")
val s2 = Student(14000, "Ana", "pt")
val s3 = Student(11000, "Ana", "en")
val cmp = ComparerOrder(Student::class)
assertTrue { cmp.compare(s1, s2) < 0 } // same nationality and 12000 is < 14000
assertTrue { cmp.compare(s2, s3) > 0 } // “pt” is > “en”
  1. You have to develop a Comparer<T : Any>(klass: KClass<T>) that implements Comparator and it is able to compare instances of type represented by klass, according to the properties which are: Comparable OR annotated with a Comparison that specifies a Comparator for that property, according to the following example:
class Person(
  val id: Int,
  val name: String,
  @Comparison(cmp = AddressByRoad::class) val address: Address,
  @Comparison(cmp = AccountByBalance::class) val account: Account) {
}

class AccountByBalance : Comparator<Account>{
  override fun compare(o1: Account, o2: Account): Int {
    return o1.balance.compareTo(o2.balance);
  }
}

class AddressByRoad : Comparator<Address> {
  override fun compare(o1: Address, o2: Address): Int {
    return o1.road.compareTo(o2.road)
  }
}
 
val p1 = Person(11000, "Ana", Address("Rua Amarela", 24), Account("FD3R", 9900))
val p2 = Person(11000, "Ana", Address("Rua Rosa", 24), Account("8YH5", 9900))
val p3 = Person(11000, "Ana", Address("Rua Rosa", 24), Account("JK2E", 100))
val p4 = Person(11000, "Ana", Address("Rua Rosa", 97), Account("BFR5", 100))
val p5 = Person(17000, "Ana", Address("Rua Rosa", 97), Account("BFR5", 100))
val cmp = Comparer<Person>(Person::class)

assertTrue { cmp.compare(p1, p2) < 0 } // Rua Amarela is < Rua Rosa
assertTrue { cmp.compare(p2, p3) > 0 } // 9900 is > 100
assertEquals(0, cmp.compare(p3, p4))   // All properties are equal
assertTrue { cmp.compare(p4, p5) < 0 } // 11000 is < 17000

Lesson 19 - Bytecode

  • Evaluations Stack
  • Local variables and Arguments
  • Constant Pool
  • 16-bit index into the constant pool
  • Load and store opcodes
  • Shortcut opcode forms
  • Arithmetic
  • Execution Flow
  • Reference Types are instantiated in bytecode with:
    • new - Allocates storage on Heap, initializes space and the object's header, and returns the reference to newbie object.
    • invokespecial - Call to class <init> method (corresponding to constructor).
  • Instantiating a refence type, e.g. Student(765134, "Ze Manel") may produce in bytecode:
new           #8   // class Student
dup                // duplicates the value on top of the stack
...                // One load (push) for each parameter of <init> (constructor)
invokespecial #14  // Method Student."<init>"

Lesson 20 - Meta-programming and Cojen Maker

  • Introduction to Metaprograming and dynamic code generation;
  • Cojen Maker API: ClassMaker, MethodMaker, FieldMaker
  • finish(): Class, finishTo(OutputStream)

Lesson 21 - DynamicMapper with Cojen Maker

  • New Dynamic Mapper that suppresses Reflect on:
    1. Getting properties from source object
    2. Instantiating the target class
  • Common base interface for NaiveMapper and Dynamic Mappers:
    • interface Mapper<T> { fun mapFrom(source: Any): T }
  • NOTE:
    • There is a single NaiveMapper class using Reflect
    • There are different Mapper classes dynamically generated for each pair srcKlass to destKlass
  • Each Dynamic Mapper has a different implementation of mapFrom.

Java Reflect <interop> Kotlin Reflect:

  • Annotation @JvmOverloads -- Instructs the Kotlin compiler to generate overloads for a function that substitute default parameter values.
  • javac -parameters - Generates metadata for reflection on method parameters.
    • Gradle : tasks.compileKotlin { kotlinOptions { javaParameters = true } }

Lesson 26 - Sequences

  • Collection Pipeline - "organize some computation as a sequence of operations which compose by taking a collection as output of one operation and feeding it into the next."
  • Advantages:
    • Composability
    • Expressivity/Readability
    • Extensibility
  • Alternative pipelines idioms:
    • e.g. method chaining:
      • students.filter(...).map(...).distinct(...).count()
    • e.g. nested function:
      • (count(remove-duplicates(mapcar #'map-function(remove-if-not ...))))
  • Collection pipeline:
    • Data Source --> Intermediate Operation* --> Terminal Operation
    • May have many intermediate operations.
    • After the terminal operation we cannot chain any intermediate operation.
  • May distinguish by:
    • Idiom of combining functions: method chain versus nested functions
    • API methods names: e.g. filter, drop, reduce, etc versus where, skip, fold, etc (in Dart)
    • Eager versus Lazy, e.g. JavaScript Array methods versus Java java.util.stream
  • Query Example: Distinct weather descriptions in rainy days.
weatherData
  .map { it.weatherDesc }
  .filter { it.lowercase().contains("rain") }
  .distinct()
weatherData
  .filter { it.weatherDesc.lowercase().contains("rain") }
  .map { it.weatherDesc }
  .distinct()
  • The order of intermediate operations in the pipeline => may impact the total number of iterations.
  • Custom implementation of an equivalent eagerMap and eagerFilter.
  • Kotlin 2 distinct APIs for collections pipeline: Iterable versus Sequence
  • Iterable and Collections are eager with horizontal processing
  • Sequence should be lazy with vertical processing and can be infinite.
  • Marble diagrams:
    • Time/Data is horizontally from left to right.
    • Operations flow vertically, representing the sequence of operations applied to the data stream
  • Homework:
    • Implement eagerDistinct.

Lesson 27 - Lazy processing

  • Implement eagerDistinct.
  • Implementing lazyMap equivalent to the lazy version of map
  • Explicit implementation of interface Iteratorfor lazyMap
  • Exercise: Implement lazyDistinct
  • Explicit implementation of interface Iteratorfor lazyDistinct

Lesson 28 - Generators and yield operator

  • Generator:
    • Like a function, but instead of returning a single value, it produces a sequence of values.
    • Uses the yield keyword to return the next item in the sequence to the caller.
    • Computation is suspended at yield and resumed when the caller requests the next item (next()).
  • A generator can remain suspended indefinitely without being resumed from its caller.
  • sequence - Builds a Sequence lazily yielding values one by one.
  • suspend fun yield(value: T) - Yields a value to the Iterator being built and suspends until the next value is requested.
  • Example: Implementing alternative suspendDistinct in 4 lines using the yield.

Exercises with lazy sequences:

  1. fun <T> Sequence<T>.concat(other: Sequence<T>) = Sequence<T>
  2. fun <T : Any?> Sequence<T>.collapse() = Sequence<T> - merges series of adjacent elements

Lesson 29 - Benchmarking

  • Benchmark - assess the relative performance
  • Benchmark != Unit Tests
  • A naif approach - direct measurement, e.g.
    • measureTimeMillis { logger.log(Student(...)) }.also { println("logger.log() took $it millis"); }
  • Some Problems:
    1. Mixing domain instantiation (i.e. Student) with operation execution logger.log().
    2. First execution includes Jitter overhead and misses optimizations.
    3. Milliseconds could not be accurate enough.
    4. IO may be orders of magnitude slower than log operation itself
    5. IDE (e.g. InteliJ) may induce other overheads.
    6. Absolute results in milliseconds may vary on different hardware environments.
    7. System.currentTimeMillis() includes a System call with implicit overhead.
    8. Garbage Collector may degrade performance
  • Minimize side effects:
    1. Remove domain instantiation from operation measurement
    2. Include warm-up => Optimizations may improve performance
    3. Measure the total execution of several iterations rather than several measurements of single executions.
    4. Avoid IO => Mocking IO
    5. Avoid extra tools such as IDE (e.g. InteliJ), gradle, or other => run directly on VM (e.g. java)
    6. Baseline => How much can we improve performance?
    7. same as 3.
    8. GC => Run several iterations and discard most divergent results.
  • JMH - Java Microbenchmark Harness.
  • Benchmark tests annotated with @Benchmark.
  • JMH Gradle Plugin
  • gradlew jmhJar
  • java -jar <path to JAR> -f 1 -wi 4 -i 4 -w 2 -r 2 -tu ms:
    • -f - forks
    • -wi - warm-up iterations
    • -i - iterations
    • -w - each warm-up iteration duration
    • -r - each iteration duration
    • -tu - time unit

Lesson 32 - Deconstructing yield and Suspend Functions

  • Continuation<T>
  • resumeWith(result: Result<Unit>)
  • Suspension points and State Machine
  • Example cast fetchSuspend(String) in fetchCps(url: String, onComplete: Continuation<String>): Any
  • Example cast fetchCps(url: String, onComplete: Continuation<String>): Any to:
    • suspend (String) -> String
  • Example state machine in fetchManyCps
  • Turn lazy in: fetchManyCpsLazy(...) : Sequence<String>
  • Implement FetchManyCpsIterator(): Continuation<String>, Iterator<String>

Lesson 33 - Lazy Iterator Builder with Suspend Functions

  • Implement FetchManySuspendIterator field nextContinuation to resume block()
  • Leverage suspend points to capture Continuation on yield.
  • Breakdown FetchManySuspendIterator bytecode:
    • TABLESWITCH
    • anonymous inner class FetchManySuspendIterator$block$1
    • FetchManySuspendIterator$block$1 field label
    • FetchManySuspendIterator$block$1 fun invokeSuspend => block()

Lesson 35 - Garbage Collector

  • Local values in Stack <versus> Objects in Heap.
  • Local variables managed on stack <versus> Objects managed on Heap by Garbage Collector
    • Stack frame on function's execution => cleaned on function completion, including local variables.
    • Unreachable objects are candidates for Garbage Collector
  • Garbage Collection - "process of looking at heap memory, identifying which objects are in use and which are not, and deleting the unused objects."
  • Unused object, or unreferenced object.
  • In use object, or a referenced object:
    • "some part of your program still maintains a pointer to that object."
    • Referenced by a root or belongs to a graph with a root reference.
    • root reference:
      • Local variables - stored in the stack of a thread.
      • Static variables - belong to the class type stored in permanent generation.
  • GC basic process: 1. Marking and 2. Deletion.
  • Deletion approaches:
    • Normal Deletion => holds a list to free spaces. !! memory allocation slower !!
    • Deletion with Compacting => move referenced objects together ++ makes new memory allocation faster !!! longer garbage collection time !!!
  • Generational Garbage Collection:
    • Most objects are short lived => GC is more effective in younger generations.
    • Enhance the performance
  • JVM Generations:
    • Young Generation is where all new objects are allocated and aged.
    • Old Generation is used to store long surviving objects.
    • Permanent generation contains metadata, classes and methods.
  • "Stop the World" events:
    • minor garbage collection - collects the young generation.
    • major garbage collection - collects the old generation.
  • Young Generation process: Eden, Survivor Space 0 and Survivor Space 1.

Lesson 36 - Closeable

Lesson 37 - Cleaner

  • Cleaner - manage cleaning actions.
    • public Cleanable register(Object obj, Runnable action) - Registers an object and a cleaning action, i.e. Runnable
    • Cleanable has a single method clean() that runs the cleaning action and unregisters the cleanable.
  • Explicitly invoke the clean() method when the object is closed or no longer needed.
  • If the close() method is not called, the cleaning action is called by the Cleaner.
Clone this wiki locally