diff --git a/Content/20230712223044-lisp.org b/Content/20230712223044-lisp.org index e72da98..09bdd6b 100644 --- a/Content/20230712223044-lisp.org +++ b/Content/20230712223044-lisp.org @@ -34,6 +34,10 @@ Maxwell's equations of Software (~Alan Kay in some Interview~) (#t (apply (car e) (evlis (cdr e) a) a)))) #+end_src * Comment-Stream +** 0x2303 + - read up on the sharp dot (#.) read macro + - https://www.lispworks.com/documentation/HyperSpec/Body/02_dhf.htm + ** 0x22DA - what goes into [[id:2dfa41e9-07ba-4deb-a466-af409fa8b465][writing a lisp]] interpreter? ** 0x227B diff --git a/Content/20230715180212-mutability.org b/Content/20230715180212-mutability.org index 480ab74..857292a 100644 --- a/Content/20230715180212-mutability.org +++ b/Content/20230715180212-mutability.org @@ -2,11 +2,4 @@ :ID: 20230715T180212.260696 :END: #+title: Mutability -#+filetags: :trait: - -* 0x211A - -The ability to be changed. If denied, we refer the said entity to be immutable. -Different styles of programming arise when one chooses a mutable/immutable representation to work with. - -This in turn gives rise to the notion of a State. +#+filetags: :programming: diff --git a/Content/20230717122922-vector_space_model.org b/Content/20230717122922-vector_space_model.org index b96bd41..93cdb42 100644 --- a/Content/20230717122922-vector_space_model.org +++ b/Content/20230717122922-vector_space_model.org @@ -1,5 +1,6 @@ :PROPERTIES: :ID: 9bb733a2-8540-4f7e-acd8-63547efa9b7e +:ROAM_ALIASES: "Spatial Data" :END: #+title: vector space model #+filetags: :math: diff --git a/Content/20230718222456-input_queue.org b/Content/20230718222456-input_queue.org index 9f5403d..f9fb986 100644 --- a/Content/20230718222456-input_queue.org +++ b/Content/20230718222456-input_queue.org @@ -28,6 +28,7 @@ These are meant to be navigated with org-roam emacs and accessing them is conven ** https://news.ycombinator.com/ ** http://wiki.c2.com/ ** https://www.cliki.net/ +** https://explodinginsights.com/ * Blogs ** The CTO Club : https://thectoclub.com/ - good for the bigger picture on emerging tech diff --git a/Content/20230912141403-functions.org b/Content/20230912141403-functions.org index 17c70a6..5820546 100644 --- a/Content/20230912141403-functions.org +++ b/Content/20230912141403-functions.org @@ -2,4 +2,4 @@ :ID: a31671c6-12ea-4fc9-93cb-73d29fd508a6 :END: #+title: Functions -#+filetags: :math: +#+filetags: :programming:math: diff --git a/Content/20231003160003-attention.org b/Content/20231003160003-attention.org index 2f1199a..0722868 100644 --- a/Content/20231003160003-attention.org +++ b/Content/20231003160003-attention.org @@ -71,37 +71,6 @@ - *[[id:799650ff-fea0-45b0-aabf-3f4d80540439][Softmax]] Scaling:* The division by \(\sqrt{d_k}\) stabilizes the gradients by preventing the dot-product values from becoming too large. - *Attention Mechanism:* It enables the transformer to focus differently on different input parts, capturing dependencies irrespective of their position. - -* Multihead Attention Detailed Elaboration - -Multihead Attention is a fundamental component of the Transformer architecture, which has been pivotal in advancing natural language processing and other fields. - -- *Core Concept*: - - Involves multiple attention mechanisms running in parallel, allowing the model to focus on different parts of the input sequence simultaneously. - -- *Multihead Attention*: - - Instead of having a single set of queries, keys, and values, Multihead Attention uses multiple sets, each called a 'head'. - - Each head performs its own scaled dot-product attention operation in parallel, and the results are concatenated and projected to obtain the final output. - - This allows the model to jointly attend to information from different representation subspaces at different positions. - -- *Benefits*: - - Enhanced modeling capacity: The use of multiple attention heads allows capturing diverse types of dependencies in the input. - - Parallelizability: Multihead attention enables parallel processing and efficient use of computational resources. - -- *Implementation Details*: - - In practice, the input is often split into a fixed number of heads, where each head performs attention on a portion of the embedding dimensions. - - The dimensions of Q, K, and V are typically chosen such that the dimensions after concatenation match the input dimension, facilitating residual connections, a key feature in Transformer architectures. - -- *Applications and Extensions*: - - Widely used in language models, such as BERT and GPT. - - Variants include multihead attention with relative positional encodings, multihead attention with local constraints in [[id:6145692e-b916-42d6-83a5-df1bc8eb2a6b][Vision Transformers]] (ViTs), etc. - -** Connections between these entities: - -- *Operational Process*: The attention mechanism's ability to focus comes from the interaction between Q, K, V, which is optimized through the training process, allowing for flexible modeling of input dependencies. -- *Architectural Design*: Multihead mechanism builds upon the basic attention to leverage positional and contextual variations that a single head may miss. -- *Use Cases*: This mechanism's applicability extends beyond NLP, making it versatile for problems involving sequential or spatial data. - * Relevant Nodes - [[id:4f9006cf-6e6f-4019-bb8d-e7d5d85e191e][Transformers]] - [[id:affff439-329d-4962-bf5f-def85d75042e][Large Language Models]] diff --git a/Content/20231012140139-large_language_models.org b/Content/20231012140139-large_language_models.org index 43314e4..aa53cb0 100644 --- a/Content/20231012140139-large_language_models.org +++ b/Content/20231012140139-large_language_models.org @@ -96,6 +96,70 @@ The scaling laws for Large Language Models (LLMs) describe how changes in differ *** Truthful QA *** Massive Multitask Language Understanding *** Word in Context -* Resources +** Hyperparameters of an LLM +*** During Training: +- *Learning Rate*: + - Controls the step size for updating model weights. + - A crucial hyperparameter as it affects convergence and stability. + +- *Batch Size*: + - Number of training examples used in one iteration. + - Larger batch sizes can stabilize gradient updates but require more memory. + +- *Number of Epochs*: + - Defines how many times the entire training dataset is passed through the model. + - Needed to ensure adequate learning without overfitting. + +- *Optimizer Type*: + - Algorithms like Adam, SGD, or RMSProp used to adjust weights. + - Different optimizers can result in varying convergence speeds and outcomes. + +- *Dropout Rate*: + - Probability of dropping units in neural networks to prevent overfitting. + - Applied to the network layers during training. + +- *Weight Initialization*: + - Strategy for initializing model weights. + - Influences how quickly and effectively the model converges. + +- *Gradient Clipping*: + - Limits the maximum value of gradients to prevent exploding gradient issues. + - Especially useful in training large networks. + +- *Warmup Steps*: + - Number of initial training steps with a gradually increasing learning rate. + - Helps avoid large sudden updates in early training. + +*** During Inference: +- *Beam Size (in beam search)*: + - Number of beams (alternate sequences) considered for output generation. + - Balances between computational resources and quality of output. + +- *Temperature*: + - Controls randomness during sampling; higher values increase randomness. + - Influences creativity versus coherence of generated text. + +- *Top-k Sampling*: + - Limits the next word selection to the top k probable entries. + - Reduces unpredictability by narrowing down the choice of words. + +- *Top-p Sampling (Nucleus Sampling)*: + - Extends top-k by choosing from a dynamically determined set of most probable outputs. + - Balances diversity and coherence more effectively than fixed k. + +- *Max Token Length*: + - Maximum number of tokens to generate in the output. + - Used to allocate computational resources appropriately. + +*** Connections: +- *Learning Rate and Warmup Steps*: + - Both influence how learning is paced and stabilized during the early training stages. + +- *Batch Size and Gradient Clipping*: + - Larger batch sizes might affect the stability of gradients, where clipping can help to prevent instabilities. + +- *Temperature, Top-k, and Top-p Sampling*: + - These hyperparameters work together to modulate the randomness and quality of the generated text during inference. + ** Book: Building LLMs for production - https://towardsai.net/book diff --git a/Content/20231116192349-the_evolution_of_knowledge.org b/Content/20231116192349-the_evolution_of_knowledge.org index 20f3af1..298ed34 100644 --- a/Content/20231116192349-the_evolution_of_knowledge.org +++ b/Content/20231116192349-the_evolution_of_knowledge.org @@ -38,3 +38,7 @@ IOU(objectivist epistemology, ontology) > IOU(epistemology, ontology) :ID: d0d75b64-13e5-4b50-8833-a2b6a47cdf19 :END: One can factorize a complex domain into its constituent abstractions by identifying the separate degrees of freedom in its description. Varying the extent of a particular degree at a time allows one to meaningfully study variants rather than simply diffing domains. + +* Generic Tools +** [[id:cbd8cb36-6cba-46c3-84bf-077756dc3b37][Etymology]] +** [[id:c30a1e1b-f615-4b02-884f-656fee743e1d][Calculi]] diff --git a/Content/20240429115034-immutability.org b/Content/20240429115034-immutability.org index 664300b..bb79740 100644 --- a/Content/20240429115034-immutability.org +++ b/Content/20240429115034-immutability.org @@ -2,5 +2,4 @@ :ID: efba8f9b-a5df-4212-94c9-230bef916b5c :END: #+title: Immutability -#+filetags: :trait: - +#+filetags: :programming: diff --git a/Content/20240630180307-video_processing.org b/Content/20240630180307-video_processing.org index 1e166f0..0b373d5 100644 --- a/Content/20240630180307-video_processing.org +++ b/Content/20240630180307-video_processing.org @@ -1,8 +1,8 @@ :PROPERTIES: :ID: c3228445-00c4-4451-a0cc-b8742673753d :END: -#+title: video processing -#+filetags: :video: +#+title: Video Processing +#+filetags: :programming:bs: * Abstract Phases of producing a video diff --git a/Content/20240630182300-ffmpeg.org b/Content/20240630182300-ffmpeg.org index c6019a6..c81a041 100644 --- a/Content/20240630182300-ffmpeg.org +++ b/Content/20240630182300-ffmpeg.org @@ -2,7 +2,7 @@ :ID: 88f03140-7c60-41aa-b191-63e1460e76bd :END: #+title: ffmpeg -#+filetags: :video:cli:tool: +#+filetags: :cli:tool: * Abstract ** FFmpeg Overview diff --git a/Content/20240707170507-handbrake.org b/Content/20240707170507-handbrake.org index 31093ad..9c3dfb9 100644 --- a/Content/20240707170507-handbrake.org +++ b/Content/20240707170507-handbrake.org @@ -2,4 +2,4 @@ :ID: 02abc3f4-0caf-481d-a439-80bf86f1740a :END: #+title: HandBrake -#+filetags: :video:gui:tool: +#+filetags: :gui:tool: diff --git a/Content/20240707170716-non_linear_editing.org b/Content/20240707170716-non_linear_editing.org index 5fcba39..c8bf07d 100644 --- a/Content/20240707170716-non_linear_editing.org +++ b/Content/20240707170716-non_linear_editing.org @@ -2,4 +2,4 @@ :ID: 4f1aa1e0-5409-4290-8564-be2e908c5c76 :END: #+title: Non-Linear Editing -#+filetags: :video: +#+filetags: :tools: diff --git a/Content/20240818165817-langchain.org b/Content/20240818165817-langchain.org index 92b8569..df7d175 100644 --- a/Content/20240818165817-langchain.org +++ b/Content/20240818165817-langchain.org @@ -32,7 +32,6 @@ ** [[id:46f09529-c273-49ed-9bf7-7e0a6d97d65c][Callbacks]] LangChain provides a callbacks system that allows you to hook into the various stages of your LLM application. This is useful for logging, monitoring, and streaming. - * LangChain Agents & Tools Overview ** What? - complete tasks using chains, prompts, memory, and tools diff --git a/Content/20240818170034-llammaindex.org b/Content/20240818170034-llammaindex.org index 0ffed02..a00af69 100644 --- a/Content/20240818170034-llammaindex.org +++ b/Content/20240818170034-llammaindex.org @@ -4,5 +4,7 @@ #+title: Llammaindex #+filetags: :ai:tool: +* Relevant Nodes +** [[id:d9d30a75-f1aa-4ca0-8480-cb617afe29ab][Deep Lake Vector Stores]] * Resources - https://www.llamaindex.ai/ diff --git a/Content/20241115120450-variable_scope.org b/Content/20241115120450-variable_scope.org new file mode 100644 index 0000000..a936142 --- /dev/null +++ b/Content/20241115120450-variable_scope.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 912ba440-5083-41a9-b2ab-553cd9a7f8c7 +:END: +#+title: Variable Scope +#+filetags: :programming: diff --git a/Content/20241115120547-referential_transparency.org b/Content/20241115120547-referential_transparency.org new file mode 100644 index 0000000..c8eb809 --- /dev/null +++ b/Content/20241115120547-referential_transparency.org @@ -0,0 +1,19 @@ +:PROPERTIES: +:ID: c6ef7bd2-dd62-44bf-90d5-a7520b873fcd +:END: +#+title: Referential Transparency +#+filetags: :programming: + +* Overview +- *Definition*: + - Referential transparency is a property of parts of a computer program where an expression can be replaced with its value without changing the program's behavior. + - often associated with [[id:20231212T081907.908301][functional programming]]. + +- *Benefits*: + - *Improved Readability and Understandability*: Because expressions can be replaced consistently with their values, it simplifies reasoning about code. + - *Easier Debugging and Testing*: With referentially transparent code, testing becomes easier as expressions yield the same result every time they are evaluated with the same inputs. + - *Facilitates Optimization*: Compilers and interpreters can more easily optimize code through techniques like [[id:1bdc93aa-b564-4520-8590-c1ffcb026f55][memoization]] or [[id:eba75a47-6e35-4146-a152-b05089bbd8ec][lazy evaluation]]. + +- *Key Concepts*: + - *Pure Functions*: Functions that are deterministic (same inputs lead to the same outputs) and do not produce [[id:ca98597a-027f-4eda-bbb2-801b8f74d0a5][side effects]], a critical component for achieving referential transparency. + - *[[id:efba8f9b-a5df-4212-94c9-230bef916b5c][Immutable]] Data*: Often accompanies referential transparency by ensuring that data is not altered after creation which supports consistency in function results. diff --git a/Content/20241115120657-memoization.org b/Content/20241115120657-memoization.org new file mode 100644 index 0000000..26134e6 --- /dev/null +++ b/Content/20241115120657-memoization.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 1bdc93aa-b564-4520-8590-c1ffcb026f55 +:END: +#+title: Memoization +#+filetags: :programming: diff --git a/Content/20241115120712-lazy_evaluation.org b/Content/20241115120712-lazy_evaluation.org new file mode 100644 index 0000000..c7b7251 --- /dev/null +++ b/Content/20241115120712-lazy_evaluation.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: eba75a47-6e35-4146-a152-b05089bbd8ec +:END: +#+title: Lazy Evaluation +#+filetags: :programming: diff --git a/Content/20241115121014-side_effects.org b/Content/20241115121014-side_effects.org new file mode 100644 index 0000000..50dce42 --- /dev/null +++ b/Content/20241115121014-side_effects.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: ca98597a-027f-4eda-bbb2-801b8f74d0a5 +:END: +#+title: Side Effects +#+filetags: :programming: diff --git a/Content/20241115121718-calculus.org b/Content/20241115121718-calculus.org new file mode 100644 index 0000000..1be6086 --- /dev/null +++ b/Content/20241115121718-calculus.org @@ -0,0 +1,32 @@ +:PROPERTIES: +:ID: c30a1e1b-f615-4b02-884f-656fee743e1d +:ROAM_ALIASES: Calculi +:END: +#+title: Calculus +#+filetags: :meta: + +* Etymological roots + +- *Origin Language*: The word "calculus" comes from Latin. +- *Latin Meaning*: In Latin, "calculus" means "small stone" or "pebble." +- *Historical Context*: + - /Counting Stones/: This term was historically used because small stones were used for counting and calculations, particularly in the ancient abacus. + - /Mathematical Practice/: As methods for systematic calculation evolved, the name stuck to represent the burgeoning mathematical field. +- *Development of Mathematical Concept*: + - /17th Century and Beyond/: The development of calculus as we understand it today is attributed to the work of mathematicians Neumann, Leibniz, and Newton in the 17th century. + - /Use as a Mathematical Term/: Over time, "calculus" began to be used more broadly in the context of mathematics to specifically signify the advanced study of rates of change and quantities found under curves. + +Connections and Further Exploration: + +- The term’s roots in counting and measurement reflect the foundational aspects of calculus as a system for analyzing changes quantitatively. +- The use of a metaphorical "small stones" indicates the fundamental building blocks nature of calculus in mathematics, similar to atoms in matter. +- The transition from physical representations (stones) to abstract mathematical formulations mirrors the broader evolution of scientific thought from tangible observations to abstract theories. + +Questions for Further Exploration: + +- How did the transition from physical methods of calculation to theoretical calculus influence other scientific domains? +- What impact did the work of Newton and Leibniz have on the understanding and teaching of calculus in modern education systems? +- How has the notion of 'building blocks' evolved in other areas of mathematics and science from the etymological roots of calculus? + +* Branches +** [[id:b2efd76b-740b-486d-b307-7ea4bed72b5d][Pi Calculus]] diff --git a/Content/20241115122742-etymology.org b/Content/20241115122742-etymology.org new file mode 100644 index 0000000..e51d985 --- /dev/null +++ b/Content/20241115122742-etymology.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: cbd8cb36-6cba-46c3-84bf-077756dc3b37 +:END: +#+title: Etymology +#+filetags: :linguistics:meta: diff --git a/Content/20241115141751-recursion.org b/Content/20241115141751-recursion.org new file mode 100644 index 0000000..923bafe --- /dev/null +++ b/Content/20241115141751-recursion.org @@ -0,0 +1,63 @@ +:PROPERTIES: +:ID: 95edc4bc-c364-4b18-833a-ba476b3283e8 +:END: +#+title: Recursion +#+filetags: :programming: + +* Overview + +- *Definition and Concept*: + - Recursion is a programming technique where a [[id:a31671c6-12ea-4fc9-93cb-73d29fd508a6][function]] calls itself, either directly or indirectly, to solve a problem. + - It typically involves a base case to stop the recursion and a recursive case to divide the problem into smaller instances of the same problem. + +- *Base Case*: + - The base case is the termination condition that stops the recursion. + - Without a base case, the recursive calls would continue indefinitely, leading to a [[id:ea557127-190f-4861-aecb-f727fe6e676b][stack overflow]]. + +- *Recursive Case*: + - This is the part of the function that includes a call to itself. + - It reduces the problem at each step, bringing it closer to the base case. + +- *Applications*: + - Used in algorithms for traversing data structures like trees and graphs (e.g., depth-first search). + - Useful in problems that can be divided into similar sub-problems, like divide and conquer algorithms (e.g., mergesort, quicksort). + +*Connections*: +- Recursion is a fundamental concept in computer science closely related to [[id:120cade5-1cf0-4afe-b541-e2b607ae77da][mathematical induction]], as both involve solving problems by breaking them down into base cases and smaller sub-problems. +- Recursion is frequently compared to [[id:40722d92-1d10-445e-bcd9-f41999ccdf52][iteration]], as both can be used to repeat tasks but have different trade-offs in terms of complexity and efficiency. + +* Interconverting Recursion and [[id:40722d92-1d10-445e-bcd9-f41999ccdf52][Iteration]] +** Steps for Interconverting Recursion to Iteration: + +1. *Identify Base Case*: + - Determine the base case(s) in recursion which serves as the termination condition. + +2. *Set Initial Conditions*: + - Translate the base case of recursion into initial conditions for the iterative loop. + +3. *Transform Recursive Logic*: + - Convert the recursive calls into loop updates. This often involves using auxiliary data structures like stacks to simulate the recursive call stack. + +4. *Loop Until Condition Met*: + - Replace the recursive function call mechanism with a loop structure to handle repetitive execution until the condition is met. + +5. *End Condition*: + - Ensure the loop has a termination condition similar to how a recursive function has a base case. + +** Steps for Interconverting Iteration to Recursion: + +1. *Define Base Case*: + - Translate the loop’s termination condition into a base case for the recursive function. + +2. *Create Recursive Function*: + - Define a recursive function that includes parameters reflecting loop variables or state. + +3. *Inject Recursive Call*: + - Replace loop iteration with a self-referential function call, where each call updates the state similar to loop iteration. + +4. *Ensure Termination*: + - Implement a termination check within the recursive function to ensure the function exits appropriately. + +* Relevant Nodes +- [[id:3a717d24-64ef-4d38-936a-6814baaa1e6a][Tail Recursion]] +- [[id:1bdc93aa-b564-4520-8590-c1ffcb026f55][Memoization]] diff --git a/Content/20241115141836-stack_overflow.org b/Content/20241115141836-stack_overflow.org new file mode 100644 index 0000000..1349299 --- /dev/null +++ b/Content/20241115141836-stack_overflow.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: ea557127-190f-4861-aecb-f727fe6e676b +:END: +#+title: stack overflow +#+filetags: :programming: diff --git a/Content/20241115141952-mathematical_induction.org b/Content/20241115141952-mathematical_induction.org new file mode 100644 index 0000000..cbe2d43 --- /dev/null +++ b/Content/20241115141952-mathematical_induction.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 120cade5-1cf0-4afe-b541-e2b607ae77da +:END: +#+title: mathematical induction +#+filetags: :math: diff --git a/Content/20241115142008-iteration.org b/Content/20241115142008-iteration.org new file mode 100644 index 0000000..967ac9a --- /dev/null +++ b/Content/20241115142008-iteration.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 40722d92-1d10-445e-bcd9-f41999ccdf52 +:END: +#+title: Iteration +#+filetags: :programming: diff --git a/Content/20241115142039-tail_recursion.org b/Content/20241115142039-tail_recursion.org new file mode 100644 index 0000000..a36e6f8 --- /dev/null +++ b/Content/20241115142039-tail_recursion.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 3a717d24-64ef-4d38-936a-6814baaa1e6a +:END: +#+title: Tail Recursion +#+filetags: :programming: diff --git a/Content/20241115142152-computational_complexity.org b/Content/20241115142152-computational_complexity.org new file mode 100644 index 0000000..f0bd107 --- /dev/null +++ b/Content/20241115142152-computational_complexity.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: bb91d3d1-c6f9-4864-9291-35c10ce79c42 +:END: +#+title: Computational Complexity +#+filetags: :programming: diff --git a/Content/20241115194144-pi_calculus.org b/Content/20241115194144-pi_calculus.org new file mode 100644 index 0000000..2a6bd50 --- /dev/null +++ b/Content/20241115194144-pi_calculus.org @@ -0,0 +1,32 @@ +:PROPERTIES: +:ID: b2efd76b-740b-486d-b307-7ea4bed72b5d +:END: +#+title: Pi Calculus +#+filetags: :meta: + +* Overview + + - for the study of [[id:618d0535-411d-4c36-b176-84413ec8bfc1][concurrent]] systems. + - allows for the description, analysis, and verification of interacting systems. + + - *Core Concepts:* + - *Processes*: The fundamental units of Pi Calculus that engage in interactions. + - *Channels*: The conduits for communication between processes. + - *Mobility*: The notion that the connection structure among processes can change over time. + - *Names*: Used in Pi Calculus to represent channels and can be communicated between processes, enabling dynamic topology. + + - *Operations:* + - *Prefix*: =(x).P= describes a process P that waits to receive input on channel x before proceeding. + - *Output*: =x.P= describes a process P that sends y over channel x and then behaves as P. + - *Parallel Composition*: =P | Q= represents the parallel execution of processes P and Q. + - *Restriction*: =νx.P= restricts the scope of the channel x to the process P, meaning x is private to P. + - *Replication*: =!P= indicates an unbounded number of copies of process P running in parallel. + + - *Applications:* + - Used in modeling and verifying protocols for [[id:a4e712e1-a233-4173-91fa-4e145bd68769][communication]] networks. + - Helps in analyzing the correctness of [[id:a3d0278d-d7b7-47d8-956d-838b79396da7][distributed systems]] and their ability to handle concurrency. + - Provides foundational structures for programming languages designed for concurrent computations. + + - *Related Concepts:* + - *Calculus of Communicating Systems (CCS)*: An earlier model of concurrency developed by Milner. + - *Mobile processes*: A shift from static to dynamic process structures, where the communication links between processes can change, a concept captured by Pi Calculus. diff --git a/Content/20241117082149-garbage_collector.org b/Content/20241117082149-garbage_collector.org new file mode 100644 index 0000000..176b94c --- /dev/null +++ b/Content/20241117082149-garbage_collector.org @@ -0,0 +1,27 @@ +:PROPERTIES: +:ID: 028bcc58-f867-432c-8623-0a40018848e6 +:ROAM_ALIASES: GC +:END: +#+title: Garbage Collector +#+filetags: :programming: + +* Overview + +** *Definition*: + - A garbage collector (GC) is a form of automatic [[id:bb936c2a-dd8a-4bab-b8f9-783c3fb54f94][memory management]]. The system is responsible for identifying and reclaiming memory that is no longer in use by programs, thus preventing [[id:5ac70a3c-79b1-44a3-bc4e-f2760f4486d6][memory leaks]]. + +** *Purpose*: + - The primary objective of garbage collection is to free up memory space that is no longer needed, enabling efficient memory usage in processes that dynamically manage memory allocation and deallocation during runtime. + +** *Mechanism*: + - *Reachability*: Typically, a GC determines object eligibility for garbage collection based on reachability; objects that cannot be reached by any reference path from a root reference are considered garbage. + - *Phases*: Common GC processes include marking, sweeping, compacting, and possibly copying. + - *Marking*: Traverses object graphs starting from root references to mark all reachable objects. + - *Sweeping*: Reclaims memory occupied by unmarked (non-reachable) objects. + - *Compacting*: Reorganizes objects in memory to reduce fragmentation and improve allocation performance. + - *Copying*: Moves reachable objects to a new memory area and reclaims the old area as free space. + +** *Types*: + - *Reference Counting*: Counts references to each object and collects objects once references reach zero. + - *Tracing Garbage Collection*: Uses graph traversal techniques (mark-and-sweep, mark-and-compact, generational) to discover unused objects. + - *Generational GC*: Optimizes for the observation that most objects die young by segregating them into different lifespan generations. diff --git a/Content/20241117082422-memory_leaks.org b/Content/20241117082422-memory_leaks.org new file mode 100644 index 0000000..1027b10 --- /dev/null +++ b/Content/20241117082422-memory_leaks.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 5ac70a3c-79b1-44a3-bc4e-f2760f4486d6 +:END: +#+title: memory leaks +#+filetags: :programming: diff --git a/Content/20241117082455-memory_management.org b/Content/20241117082455-memory_management.org new file mode 100644 index 0000000..6225441 --- /dev/null +++ b/Content/20241117082455-memory_management.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: bb936c2a-dd8a-4bab-b8f9-783c3fb54f94 +:END: +#+title: memory management +#+filetags: :programming: diff --git a/Content/20241120123059-deep_lake_vector_store.org b/Content/20241120123059-deep_lake_vector_store.org new file mode 100644 index 0000000..c152027 --- /dev/null +++ b/Content/20241120123059-deep_lake_vector_store.org @@ -0,0 +1,8 @@ +:PROPERTIES: +:ID: d9d30a75-f1aa-4ca0-8480-cb617afe29ab +:END: +#+title: deeplake.ai +#+filetags: :ai:data: + +* Resources + - https://www.deeplake.ai/ diff --git a/Content/20241120140116-data_lineage.org b/Content/20241120140116-data_lineage.org new file mode 100644 index 0000000..ba7100f --- /dev/null +++ b/Content/20241120140116-data_lineage.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: f216db87-404d-446e-9157-89d8bb76b4e6 +:END: +#+title: Data Lineage +#+filetags: :data: diff --git a/Content/20241120141611-vector_store_index.org b/Content/20241120141611-vector_store_index.org new file mode 100644 index 0000000..002e98e --- /dev/null +++ b/Content/20241120141611-vector_store_index.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 9c288e43-1136-4bf5-9d12-26ada642224b +:END: +#+title: Vector Store Index +#+filetags: :data:ai: diff --git a/Content/20241120151630-kd_trees.org b/Content/20241120151630-kd_trees.org new file mode 100644 index 0000000..7299ba0 --- /dev/null +++ b/Content/20241120151630-kd_trees.org @@ -0,0 +1,38 @@ +:PROPERTIES: +:ID: a6434778-8ee0-4560-8637-5c583dcd85ec +:END: +#+title: KD Trees +#+filetags: :programming:data: + +* Overview + - A k-dimensional tree, known as a KD Tree, is a [[id:20230715T173535.681936][data structure]] commonly used for organizing points in a k-dimensional [[id:9bb733a2-8540-4f7e-acd8-63547efa9b7e][space]]. + - Primarily used for applications involving multidimensional [[id:656af4b9-648b-41f9-932b-cbf2d2017794][search]] queries such as range searches and [[id:26c1717a-55a6-43d7-8a92-d39800e31df1][nearest neighbor searches]]. + + - *Key Characteristics*: + - *[[id:3821a4f5-998a-4903-970f-d95bf2ed8cd4][Binary Tree]]*: Similar to a binary search tree but designed for multiple dimensions. + - *Recursive Partitioning*: The space is recursively divided into two half-spaces at each step by a [[id:892fad58-dbf7-4db6-8184-51bd4276d462][hyperplane]]. + - *Non-balanced*: It may not be balanced; balance depends on the order of input data. + + - *Construction*: + - *Inserting Elements*: Splits are done along one dimension at each level of the tree, typically cycling through the dimensions. + - *Depth*: A balanced KD Tree with =n= points will have a depth that is typically =O(log n)=. + + - *Usage*: + - *Range Search*: Efficiently find all points within a specific range in a multidimensional space. + - *Nearest Neighbor Search*: Locate the nearest data point to a given query point in =O(log n)= time on average for balanced trees. + + - *Algorithms*: + - *Build*: Typically built in =O(n log n)= time complexity where =n= is the number of points. + - *Query*: Queries like nearest neighbor and range queries typically have an average log-scale complexity with respect to the size of the dataset. + + - *Limitations*: + - *[[id:848387f1-ec7a-4410-9ddd-40849472ef1d][Curse of Dimensionality]]*: Efficiency decreases as the number of dimensions increases, diminishing its advantage over brute-force methods. + - *Imbalance*: Poor insertion order can lead to imbalanced trees, potentially degrading performance. + +*Connections and Pathways*: +- How does recursive partitioning influence the searching times in a KD Tree? +- What are the strategies to ensure a more balanced KD Tree? +- How do alternative data structures compare to KD Trees for higher-dimensional data, and at what dimensionality do they become more efficient? +- In what specific scenarios is a KD Tree more practical compared to other spatial data structures like quad-trees or [[id:9f869b16-1bf8-485b-a878-6d00107848ec][R-trees]]? + +If you need more detailed information on a specific aspect of KD Trees or have a particular example in mind, could you please provide more context? This would help in tailoring the details more precisely. diff --git a/Content/20241120151737-nearest_neighbor_searches.org b/Content/20241120151737-nearest_neighbor_searches.org new file mode 100644 index 0000000..efc43b5 --- /dev/null +++ b/Content/20241120151737-nearest_neighbor_searches.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 26c1717a-55a6-43d7-8a92-d39800e31df1 +:END: +#+title: Nearest Neighbor Searches +#+filetags: :ai:programming: diff --git a/Content/20241120151833-binary_tree.org b/Content/20241120151833-binary_tree.org new file mode 100644 index 0000000..269badc --- /dev/null +++ b/Content/20241120151833-binary_tree.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 3821a4f5-998a-4903-970f-d95bf2ed8cd4 +:END: +#+title: Binary Tree +#+filetags: :data:programming: diff --git a/Content/20241120151850-hyperplane.org b/Content/20241120151850-hyperplane.org new file mode 100644 index 0000000..c987d00 --- /dev/null +++ b/Content/20241120151850-hyperplane.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 892fad58-dbf7-4db6-8184-51bd4276d462 +:END: +#+title: hyperplane +#+filetags: :math: diff --git a/Content/20241120152041-r_trees.org b/Content/20241120152041-r_trees.org new file mode 100644 index 0000000..72dca18 --- /dev/null +++ b/Content/20241120152041-r_trees.org @@ -0,0 +1,42 @@ +:PROPERTIES: +:ID: 9f869b16-1bf8-485b-a878-6d00107848ec +:END: +#+title: R-Trees +#+filetags: :programming:data: + +* Overview + +- *Definition*: + - R-Trees are a type of data structure that serves as an efficient spatial access method, often used to index multi-dimensional information, such as geographical coordinates. + - They are particularly useful for spatial queries like range searches and [[id:26c1717a-55a6-43d7-8a92-d39800e31df1][nearest neighbor searches]]. + +- *Structure*: + - The R-Tree structure is hierarchical and tree-like, with a root node that branches into child nodes. + - Nodes consist of bounding rectangles, typically known as minimum bounding rectangles (MBRs), which encapsulate their child rectangles. + - Each node in an R-Tree represents a spatial region, and the leaf nodes contain pointers to the actual spatial data. + +- *Operations*: + - *Insertion*: Adding an item involves determining the appropriate leaf where the item’s MBR can be added with the least overlap. + - *Deletion*: Removing an item requires re-adjustment of the tree structure to maintain the properties of the R-Tree. + - *Search*: R-Tree searches involve traversing the tree, starting from the root, and checking which nodes intersect with the search area. + +- *Variants*: + - *R/-Trees*: An enhanced version of R-Trees aimed at reducing overlap of the MBRs, thereby improving query performance. + - *R+ Trees*: Another variant that avoids overlap in leaf nodes, making insertion and deletion more complex. + +- *Applications*: + - [[id:d3e56924-5a2e-4b5a-8866-d9f689ca21c0][Geographic Information Systems]] (GIS) for managing [[id:9bb733a2-8540-4f7e-acd8-63547efa9b7e][spatial data]]. + - [[id:64f87394-d866-452e-b1c0-0f4a9e817424][Computer graphics]] for collision detection. + - Database systems for multi-dimensional queries. + +*Connections between Entities*: + +- R-Trees stand as a bridge between the theoretical concept of spatial indexing and practical applications in fields requiring multidimensional data querying. +- Variants like R/-Trees and R+ Trees address specific inefficiencies in the basic R-Tree design, demonstrating the iterative nature of technological development. + +*Further Considerations*: + +- How do the performance characteristics of R-Trees compare with other spatial data structures, such as k-d trees or quad-trees? +- What are the theoretical limitations of R-Trees in terms of data dimensionality and size? + +Which specific aspect of R-Trees or their applications would you like to explore further? diff --git a/Content/20241120152253-geographic_information_systems.org b/Content/20241120152253-geographic_information_systems.org new file mode 100644 index 0000000..26813b0 --- /dev/null +++ b/Content/20241120152253-geographic_information_systems.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: d3e56924-5a2e-4b5a-8866-d9f689ca21c0 +:END: +#+title: Geographic Information Systems +#+filetags: :programming:geography: diff --git a/Content/20241120152352-computer_graphics.org b/Content/20241120152352-computer_graphics.org new file mode 100644 index 0000000..9d4afc7 --- /dev/null +++ b/Content/20241120152352-computer_graphics.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 64f87394-d866-452e-b1c0-0f4a9e817424 +:END: +#+title: Computer Graphics +#+filetags: :programming: diff --git a/Content/20241120161138-vantage_point_trees.org b/Content/20241120161138-vantage_point_trees.org new file mode 100644 index 0000000..1cf9afc --- /dev/null +++ b/Content/20241120161138-vantage_point_trees.org @@ -0,0 +1,32 @@ +:PROPERTIES: +:ID: ecdf53f5-9659-4ea4-9fe7-b9035a83a918 +:ROAM_ALIASES: "VP Trees" +:END: +#+title: Vantage Point Trees +#+filetags: :data:programming: + +* Overview + - *Data Structure*: VP Trees are a type of metric tree used for [[id:26c1717a-55a6-43d7-8a92-d39800e31df1][nearest neighbor search]] in [[id:9bb733a2-8540-4f7e-acd8-63547efa9b7e][metric spaces]]. + - *Purpose*: They efficiently handle queries like "find the closest data points" by organizing data points in a way that reduces the number of distance calculations needed. + - *Use Case*: Commonly used in applications like image retrieval, fingerprint recognition, and any domain where quick nearest neighbor search is required. + + - *Construction* + - *Vantage Point Selection*: A 'vantage point' is chosen from the dataset (typically random). + - *Partitioning*: Data points are divided into two groups based on their distance from the vantage point. This involves selecting a radius around the vantage point and classifying points into "inside" and "outside" groups. + - *Recursive Application*: This process is recursively applied to the groups, constructing the binary tree. + + - *Query Execution* + - *Nearest Neighbor Search*: Searches are performed by traversing the tree, using the distances from the vantage points at each level to efficiently exclude large portions of data from search. + - *Pruning Capability*: During the search, branches that cannot contain the nearest neighbor are pruned, allowing for efficient query processing. + + - *Advantages* + - *Efficiency*: Can significantly reduce the number of distance computations compared to a linear search, especially in high-dimensional data. + - *Adaptability*: Works well with diverse types of metric spaces, not limited to Euclidean distance. + + - *Challenges* + - *High Dimensionality*: While VP Trees improve search times, they are still affected by the “[[id:848387f1-ec7a-4410-9ddd-40849472ef1d][curse of dimensionality]]” common in all metric trees. + - *Dynamic Updates*: Insertion and deletion of data can be complex as it may require tree reconstruction. + +*Connections:* +- *Metric Trees*: VP Trees are one of several types of metric trees, similar in purpose to KD-trees but more suited to [[id:70f4b746-2255-40ab-b63e-aa64630890dc][non-Euclidean]] spaces. +- *Nearest Neighbor Algorithms*: Relate closely with other methods such as [[id:a6434778-8ee0-4560-8637-5c583dcd85ec][KD-trees]], [[id:949f8091-e900-4e99-9a1b-c02e6b9d4d3d][Ball Trees]], and [[id:5a86e5bf-214c-44c2-9668-aa5c965b8ec1][Cover Trees]], each with strengths and weaknesses depending on the context. diff --git a/Content/20241120162157-non_euclidean_space.org b/Content/20241120162157-non_euclidean_space.org new file mode 100644 index 0000000..f2ac161 --- /dev/null +++ b/Content/20241120162157-non_euclidean_space.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 70f4b746-2255-40ab-b63e-aa64630890dc +:END: +#+title: Non-Euclidean Space +#+filetags: :geometry: diff --git a/Content/20241120162736-ball_trees.org b/Content/20241120162736-ball_trees.org new file mode 100644 index 0000000..4e778d8 --- /dev/null +++ b/Content/20241120162736-ball_trees.org @@ -0,0 +1,7 @@ +:PROPERTIES: +:ID: 949f8091-e900-4e99-9a1b-c02e6b9d4d3d +:END: +#+title: Ball Trees +#+filetags: :programming:data: + +* Overview diff --git a/Content/20241120162751-cover_trees.org b/Content/20241120162751-cover_trees.org new file mode 100644 index 0000000..14448a2 --- /dev/null +++ b/Content/20241120162751-cover_trees.org @@ -0,0 +1,5 @@ +:PROPERTIES: +:ID: 5a86e5bf-214c-44c2-9668-aa5c965b8ec1 +:END: +#+title: Cover Trees +#+filetags: :programming:data: diff --git a/Content/20241120164559-hugging_face.org b/Content/20241120164559-hugging_face.org new file mode 100644 index 0000000..1eca993 --- /dev/null +++ b/Content/20241120164559-hugging_face.org @@ -0,0 +1,9 @@ +:PROPERTIES: +:ID: b6db2d75-f3f9-454b-a847-b22446401c39 +:END: +#+title: Hugging Face +#+filetags: :tool:ai: + + +* Resources + - https://huggingface.co/docs/hub/en/index diff --git a/Content/20241120165903-cohere.org b/Content/20241120165903-cohere.org new file mode 100644 index 0000000..30da581 --- /dev/null +++ b/Content/20241120165903-cohere.org @@ -0,0 +1,8 @@ +:PROPERTIES: +:ID: f23475aa-a639-4550-819d-5c2201b22fbe +:END: +#+title: Cohere +#+filetags: :tool:ai: + +* Resources + - https://cohere.com/