Skip to content

lessons

Jump to bottom
Miguel Gamboa edited this page Jun 15, 2022 · 28 revisions

Lessons:

Lesson 01 - Introduction and Modern VMs, e.g. JVM, Node

  • Bibliography and Lecturing methodology: github and slack.
  • Tools: kotlinc, JDK 17 and gradle.
  • Program outline in 3 parts:
    • Kotlin Type System and Reflection;
    • JVM Type System and Metaprogramming;
    • Iterators versus Sequences (yield). Generics and Reified type parameters.
  • Project in 3 parts according to program outline.
  • Managed Runtime or Execution Environment.
  • Informally virtual machine (VM) or runtime
  • Historic evolution since Java to nowadays
  • Examples of languages targeting JVM: Java, kotlin, Scala, Clojure.
  • Examples of languages targeting Node: JavaScript, TypeScript, Kotlin.
  • Java syntax similar to C and C++.
  • Distinguish between Programming Language <versus> VM
  • Managed Runtime:
    • Software components = Metadata + IR (intermediate representation) (e.g. bytecodes)
    • Portability - Compile once and run everywhere.
    • Jitter - Just-in-time compiler
    • Safety - NO invalid memory accesses
    • GC - Garbage Collector.
    • ClassLoader and CLASSPATH - dynamic and lazy load.
  • CLASSPATH
  • Interoperability Java <> Kotlin supported by JVM.
  • Using javap -c -p Point.class to inspect metadata and bytecodes definition of data class Point
    • Kotlin properties x, y and module map to methods getX(), getY() and getModule().

Lesson 02 - Kotlin Type System

  • Type System - Set of rules and principles that specify how types are defined and behave.
  • Types are defined through Classes or Interfaces
    • On JVM types are defined through Classes.
  • Classes have Members.
  • Members in JVM may be Fields or Functions (aka as methods).
  • Members in Kotlin may be Properties or Functions.
  • There are NO Properties at JVM level.
  • A Kotlin property may generate:
    • Backing field -- accessed with getfield bytecode.
    • Getter, i.e. function get... -- called with invoke... bytecode.
    • Setter, i.e. function set... (if defined with var).
  • Kotlin val <=> Java final.
  • Any <=> Java Object, which is the base class of all classes.
  • Implicit call to base constructor ...: Any():
    • bytecodes: invokespecial Object.<init>()
  • new in Java and new in bytecodes.
  • new is implicit in Kotlin.

Lesson 03 - Components and Metadata

  • Component - Reusable software unit, with:
    • IR - code in intermediate representation (e.g. bytecodes, IL, other)
    • Metadata - auto description
    • Ready to use => does not require static compilation.
    • API => conforms to a public interface.
    • Indivisible => 1 module (file)
  • Software development by Components:
    • Reduce Complexity;
    • Promote Reuse.
  • Developer roles:
    • Provider - provides a component with a well-known API (e.g. Point)
    • Client - uses the component from a provider to build an Application, or another component (e.g. App).
  • Unmanaged <versus> Managed
  • Static <versus> Dynamic link
  • Building unmanaged components with static link:
    • NOT a unit, but two parts instead: header + obj
    • Need of a header file that describes the component content
    • Different builds for different architectures
    • Structural modifications (new header) => compilation + link
    • Behavioral modifications => link
  • Demo with an unmanaged App using a Point.
  • Demo with a managed App using a Point.
    • Jitter (just-in-time compiler) - compiles IR to native code (e.g. x86, amd64, ppc) at runtime
    • Dynamic link - Point.class on App compilation + link at runtime.
    • Lazy Load - Point.class is loaded only when needed

Lesson 04 - Programming with Metadata

  • Reflection object oriented API for metadata
  • Kotlin Reflection API (kotlin.reflect): KClass, KCallable, KFunction.
  • Java Reflection API (java.lang.reflect): Class, Member, Method and Field.
  • KClass is representing a type in Kotlin, whereas Class is representing a type in JVM.
  • KClass --- .java ---> Class
  • Class ---- .kotlin ---> KClass
  • ClassLoader and URLClassLoader
  • loadClass(): Class
  • Closeable interface and Kotlin extension use()

Lesson 05 - Programming with Metadata

  • Inspecting functions trough Reflection API
  • KFunction properties: name, type, parameters and instanceParameter.
  • KFunction ----->* KParameter
  • KParameter property kind: INSTANCE versus EXTENSION_RECEIVER
  • KParameter property isOptional
  • KClass::createInstance()
  • KFunction::call()
  • Collections extensions: map, filter, forEach
  • Implementing a ZipInputStream extension iterable() with an explicit Iterator implementation.

Lesson 06 - Logger

  • Distinguishing the role of App domain ---> Logger utility.
  • Unit tests.
  • Code coverage.
  • Decouple the output into a Printer interface.
  • Implement PrinterConsole and PrinterStringBuffer.
  • class Logger(val out: Printer = PrinterConsole())

Lesson 07 - Logger and interface Getter

  • Refactor Logger to avoid if/else validation on log() execution.
  • Maintain getters on MutableMap<KClass<*>, List<Getter>>
  • interface Getter {fun readAndPrint(target: Any) }
  • Distinct implementations: createGetterProperties, createGetterFunctions, ...

Lesson 10 - Annotations

Lesson 13 - Reflection Exercises

  • fun KClass<*>.isSubclassOf(base: KClass<*>): Boolean
  • fun KClass<*>.isSuperclassOf(derived: KClass<*>): Boolean

  1. You have to develop a ComparerOrder(klass: KClass<Any>) that implements Comparator and it is able to compare instances of type represented by klass, according to the properties which are both: Comparable and annotated with Comparison. Notice that you should compare respecting the order sepcified in annotation. Example:
class Student (
  @Comparison(2) val nr:Int,
  val name: String,
  @Comparison(1) val nationality: String,
)
 
val s1 = Student(12000, "Ana", "pt")
val s2 = Student(14000, "Ana", "pt")
val s3 = Student(11000, "Ana", "en")
val cmp = ComparerOrder(Student::class)
assertTrue { cmp.compare(s1, s2) < 0 } // same nationality and 12000 is < 14000
assertTrue { cmp.compare(s2, s3) > 0 } // “pt” is > “en”
  1. You have to develop a Comparer(klass: KClass<Any>) that implements Comparator and it is able to compare instances of type represented by klass, according to the properties which are: Comparable OR annotated with a Comparison that specifies a Comparator for that property, according to the following example:
class Person(
  val id: Int,
  val name: String,
  @Comparison(cmp = AddressByRoad::class) val address: Address,
  @Comparison(cmp = AccountByBalance::class) val account: Account) {
}

class AccountByBalance : Comparator<Account>{
  override fun compare(o1: Account, o2: Account): Int {
    return o1.balance.compareTo(o2.balance);
  }
}

class AddressByRoad : Comparator<Address> {
  override fun compare(o1: Address, o2: Address): Int {
    return o1.road.compareTo(o2.road)
  }
}
 
val p1 = Person(11000, "Ana", Address("Rua Amarela", 24), Account("FD3R", 9900))
val p2 = Person(11000, "Ana", Address("Rua Rosa", 24), Account("8YH5", 9900))
val p3 = Person(11000, "Ana", Address("Rua Rosa", 24), Account("JK2E", 100))
val p4 = Person(11000, "Ana", Address("Rua Rosa", 97), Account("BFR5", 100))
val p5 = Person(17000, "Ana", Address("Rua Rosa", 97), Account("BFR5", 100))
val cmp = Comparer<Person>(Person::class)

assertTrue { cmp.compare(p1, p2) < 0 } // Rua Amarela is < Rua Rosa
assertTrue { cmp.compare(p2, p3) > 0 } // 9900 is > 100
assertEquals(0, cmp.compare(p3, p4))   // All properties are equal
assertTrue { cmp.compare(p4, p5) < 0 } // 11000 is < 17000

Lesson 16 - Dynamic code E.g. JavaPoet

  • JavaPoet API

Lesson 17 - LoggerDynamic

  • Refactoring Logger with new:
    • GetterProperty and GetterFunction implementations of interface Getter
    • AbstactLogger base class of LoggerReflect and LoggerDynamic
  • Skeleton for buildGetterProperty(klass: KClass<*>, prop: KProperty<*>) that builds a custom implementation of Getter for given domain klass and property prop.

Homework 5

Lesson 20 - LoggerDynamic

  • Finishing buildGetterProperty() and buildGetterFunc().
  • Finishing LoggerDynamic and unit tests.
  • Benchmark - assess the relative performance
  • Benchmark != Unit Tests
  • Minimize side effects:
    • Garbage Collector may degrade performance => Run several iterations and discard most divergent results.
    • Optimizations may improve performance => Include warm-up.
    • Avoid IO
    • Avoid extra tools such as IDE (e.g. InteliJ), gradle, or other => run directly on VM (e.g. java)
  • JMH - Java Microbenchmark Harness.
  • Benchmark tests annotated with @Benchmark.
  • JMH Gradle Plugin
  • gradlew jmhJar
  • java -jar <path to JAR> -f 1 -wi 4 -i 4 -w 2 -r 2 -tu ms:
    • -f - forks
    • -wi - warm-up iterations
    • -i - iterations
    • -w - each warm-up iteration duration
    • -r - each iteration duration
    • -tu - time unit
Benchmark                            Mode   Cnt     Score     Error   Units
benchLogSavingsAccountViaDynamic     thrpt    4  8511,015   254,220  ops/ms
benchLogSavingsAccountViaReflection  thrpt    4  4823,931   160,847  ops/ms
  • Analyze bytecodes of Getter regarding Object v = ((SavingsAccount) target).getBalance();
  • Equivalent to:
long bal = ((SavingsAccount) target).getBalance();
Object v = bal; // IMPLICIT call to Long.valueOf(long)
  • Primitive <versus> Reference types
  • int, long, double, ... <versus> Integer, Long, Double, ...
  • Local values in Stack <versus> Values in Heap.
  • Autoboxing is the automatic conversion that the Java compiler makes between the primitive types and their corresponding object wrapper classes.
  • unboxing - converting an object of a wrapper type (e.g. Integer) to its corresponding primitive (int) value is called .

Homework 6

Lesson 23 - Types and Objects at runtime

  • Categories of Types:
    • Value Types (e.g. Primitive types, e.g. Java int, long, double,...)
    • Reference Types (e.g. class, interface,...)
  • Values <versus> Objects (in Heap);
  • Local variables managed on stack <versus> Objects managed on Heap by Garbage Collector
    • Stack frame on function's execution => cleaned on function completion, including local variables.
    • Unreachable objects are candidates for Garbage Collector
  • Object's layout and header (containing type information):
    • header is useful for operations such as obj::class (<=> obj.getClass() in Java)
  • Reference Types are instantiated in bytecodes with:
    • new - Allocates storage on Heap, initializes space and the object's header, and returns the reference to newbie object.
    • invokespecial - Call to class <init> method (corresponding to constructor).
  • Instantiating a refence type, e.g. Student(765134, "Ze Manel") may produce in bytecodes:
new           #8   // class Student
dup                // duplicates the value on top of the stack
...                // One load (push) for each parameter of <init> (constructor)
invokespecial #14  // Method Student."<init>"
  • Casting between reference types:
    • upcasting in bytecodes it is only copying references, e.g. aload_0 and astore_1
    • downcasting includes checkcast bytecode
  • checkcast checks for compatibility (i.e. same type or subtype).
  • checkcast throws ClassCastException on unsuccess, otherwise leaves the object's reference on top of the stack.
  • Primitive <-> Reference: boxing or unboxing:
    • There is no specific JVM bytecode for these conversions.
    • Supported in JVM through auxiliary functions (i.e. valueOf() and <type>Value()) of Wrapper classes.
    • E.g. in .Net the IL language (<=> bytecodes) there are opcodes for box and unbox.
  • Primitive Type -> Reference: boxing through <Wrapper Type>.valueOf(primitive)
    • e.g. fun boxing(nr: Int) : Any { return nr }
  • Reference Type -> Primitive: unboxing through <Wrapper>.<primitive>Value()
    • e.g. fun unboxing(nr: Int?) : Int { return nr ?: throw Exception("Null not supported for primitive Int!") }
Types Java Kotlin
Primitive int, long, double, ... Int, Long, Double, ...
Reference Integer, Long, Double, ... Int?, Long?, Double?, ...

Lesson 24 - Optimizing LoggerDynamic to avoid boxing

Object v = ((SavingsAccount) target).getBalance(); // boxing: Integer.valueOf(...)
out.print("balance = " + v + ")");
  • 1st approach - replace Object by int 
int v = ((SavingsAccount) target).getBalance();
out.print("balance = " + v + ")");
  • Yet, the second statement still requires boxing v.
  • 2nd approach - overload print(<primitive>) for each primitive type:
int v = ((SavingsAccount) target).getBalance();
out.print("balance = "); // invoke Printer::print(String)
out.print(v);            // invoke Printer::print(int)
out.print(")");          // invoke Printer::print(String)
  • Speedup from 2x to 3x on LoggerDynamic for domain objects with primitive fields:
Benchmark                            Mode   Cnt     Score     Error   Units
benchLogSavingsAccountViaDynamic     thrpt    4  15069,625  7535,855  ops/ms
benchLogSavingsAccountViaReflection  thrpt    4   4705,736   753,258  ops/ms

Lesson 26 - Sequences, Iterable and Generics

  • Collection Pipeline - "organize some computation as a sequence of operations which compose by taking a collection as output of one operation and feeding it into the next."
    • e.g. method chaining: sudents.filter(...).map(...).distinct(...).count()
    • e.g. nested function: count(distinct( map (... filter (...))))
  • Collection pipeline:
    • Data Source --> Intermediate Operation* --> Terminal Operation
    • May have many intermediate operations.
    • After the terminal operation we cannot chain any intermediate operation.
  • Similarities to SQL for relational data, e.g. SELECT COUNT (DISTINCT ...) FROM ... WHERE ...
  • Distinguishing between:
    • General-purpose language, e.g. Kotlin, Java C#,
    • Domain-specific languages, e.g. SQL, Linq, Jetpack Compose,
  • Query Example: From a students listing select the first surname starting with letter A of a student with number greater than 47000.
  • Divide and reorder requirements:
    • From a students listing... - data source
    • ... of a student with number greater than 47000 - - intermediate operation
    • ... surname ... - intermediate operation
    • ... starting with letter A... - - intermediate operation
    • ... select the first... - terminal operation
  • One operation for each requirement:
File(…).readLines().parseCsv()  // List<Map<String, String>>
  .convertToStudent()           // List<Student>
  .whereNrGreaterThan(47000)    // List<Student>
  .convertToSurname()           // List<String>
  .whereStartsWith(“A”)         // List<String>
  .iterator().next()            // Select first occurence
  • Approach 1 problems:
    1. Eager processing overhead => traverses all elements regardless the number of selected items (i.e. first occurrence in this example).
    2. GC overhead => each intermediate operation stores all elements in an auxiliary List that is candidate for GC after operation's completion.
    3. Repeated code between convert... and where...
    4. Mixing Domain and Utility features
  • Approach 2 - implement generic methods convert and where:
    • fun <T, R> Iterable<T>.convert(transform: (T) -> R): Iterable<R>
    • fun <T> Iterable<T>.where(predicate: (T) -> Boolean): Iterable<T>
  • Eager implementation of select similar to mapTo of Collections Extensions:
fun <T, R, C : MutableCollection<in R>> Iterable<T>.mapTo(destination: C, transform: (T) -> R): C {
    for (item in this)
        destination.add(transform(item))
    return destination
}
  • Eager implementation of filter similar to filterTo of Collections Extensions:
public inline fun <T, C : MutableCollection<in T>> Iterable<T>.filterTo(destination: C, predicate: (T) -> Boolean): C {
    for (element in this)
        if (predicate(element))
           destination.add(element)
    return destination
}

Lesson 27 - Inline functions and Reified type parameters

  • Approach 3 - Implement a generic auxiliary toObject():
    • fun <T : Any> Map<String, String>.toObject(destination: KClass<T>) : T
    • that replaces the use of fun Map<String, String>.toStudent() : Student
  • Type Erasure:
    • Replace all type parameters in generic types with their bounds or Object
    • Insert type casts (i.e. bytecode checkcast) if necessary to preserve type safety.
  • Type Erasure and the need of parameter destination: KClass<T>
  • Inline functions and Reified type parameters to allow the use of T::class

Lesson 28 - Iterable<T> lazy implementation and Sequence yield

  • Approach 4 -Iterable<T> lazy implementation of convertLazy and filterLazy
  • Comparing the number of calls to transform between an eager and lazy pipeline.
  • Approach 5 - sequence builder that lazily yields values one by one.
fun <T, R> Sequence<T>.convert(transform: (T) -> R) = sequence {
    for (item in this@convert) {
        yield(transform(item))
    }
}

Lesson 29 - Exercises with lazy sequences

  1. fun <T> Sequence<T>.concat(other: Sequence<T>) = Sequence<T>
  2. fun <T : Any?> Sequence<T>.collapse() = Sequence<T>
  3. fun <T> Sequence<T>.window(size: Int) = Sequence<Sequence<T>>

Lesson 32 - Garbage Collection, AutoCloseable and Cleaner

  • Garbage Collection - "process of looking at heap memory, identifying which objects are in use and which are not, and deleting the unused objects."
  • Unused object, or unreferenced object.
  • In use object, or a referenced object:
    • "some part of your program still maintains a pointer to that object."
    • Referenced by a root or belongs to a graph with a root reference.
    • root reference:
      • Local variables - stored in the stack of a thread.
      • Static variables - belong to the class type stored in permanent generation.
  • GC basic process: 1. Marking and 2. Deletion.
  • Deletion approaches:
    • Normal Deletion => holds a list to free spaces. !! memory allocation slower !!
    • Deletion with Compacting => move referenced objects together ++ makes new memory allocation faster !!! longer garbage collection time !!!
  • Generational Garbage Collection:
    • Most objects are short lived => GC is more effective in younger generations.
    • Enhance the performance
  • JVM Generations:
    • Young Generation is where all new objects are allocated and aged.
    • Old Generation is used to store long surviving objects.
    • Permanent generation contains metadata, classes and methods.
  • "Stop the World" events:
    • minor garbage collection - collects the young generation.
    • major garbage collection - collects the old generation.
  • Young Generation process: Eden, Survivor Space 0 and Survivor Space 1.
  • Cleaner - manage cleaning actions.
  • Explicitly invoke the clean() method when the object is closed or no longer needed.
  • If the close() method is not called, the cleaning action is called by the Cleaner.

Lesson 35 - Cleaner

  • Cleaner - manage cleaning actions.
    • public Cleanable register(Object obj, Runnable action) - Registers an object and a cleaning action, i.e. Runnable
    • Cleanable has a single method clean() that runs the cleaning action and unregisters the cleanable.
  • Explicitly invoke the clean() method when the object is closed or no longer needed.
  • If the close() method is not called, the cleaning action is called by the Cleaner.
Clone this wiki locally