From d02b93715932e75675869af3c7333698e2069476 Mon Sep 17 00:00:00 2001
From: JadenFiottoKaufman <JadenFK@outlook.com>
Date: Sun, 3 Dec 2023 02:06:16 -0500
Subject: [PATCH] About page updates

---
 public/_sources/about.rst.txt                 |  33 ++++++++-------
 public/about/index.html                       |  39 ++++++++++--------
 public/objects.inv                            | Bin 3713 -> 3715 bytes
 public/searchindex.js                         |   2 +-
 .../notebooks/token_indexing/index.html       |   4 +-
 5 files changed, 44 insertions(+), 34 deletions(-)

diff --git a/public/_sources/about.rst.txt b/public/_sources/about.rst.txt
index ced8e19..f394641 100644
--- a/public/_sources/about.rst.txt
+++ b/public/_sources/about.rst.txt
@@ -1,11 +1,16 @@
 
-About nnsight: a transparent science API for black-box inference
-================================================================
+About nnsight
+=============
+
+An API for transparent science on black-box AI
+----------------------------------------------
 
 .. card:: How can you study the internals of a deep network that is too large for you to run?
 
-    In this era of large-scale deep learning, the most interesting AI models are massive and hard to run.
-    Ordinary commercial inference service APIs let you interact with huge models, but they do not let you see model internals.
+    In this era of large-scale deep learning, the most interesting AI models are massive black boxes
+    that are hard to run. Ordinary commercial inference service APIs let you interact with huge
+    models, but they do not let you see model internals.
+
     The nnsight library is different: it gives you full access to all the neural network internals.
     When used together with a remote service like the National Deep Inference Facility (NDIF),
     it lets you run expriments on huge open models easily, with full transparent access.
@@ -13,7 +18,7 @@ About nnsight: a transparent science API for black-box inference
 
 .. figure:: _static/images/remote_execution.png
 
-    An overview of the nnsight/NDIF pipeline. Researchers write simple Python code to run along with the neural network locally or remotely. Unlike with commercial inference, the experiment code can read or write any of the internal states of the neural networks being studied.  This code creates a computation graph that is sent to the remote service (if needed) and then interleaved with the execution of the neural network.
+    An overview of the nnsight/NDIF pipeline. Researchers write simple Python code to run along with the neural network locally or remotely. Unlike commercial inference, the experiment code can read or write any of the internal states of the neural networks being studied.  This code creates a computation graph that can be sent to the remote service and interleaved with the execution of the neural network.
 
 How you use nnsight
 -------------------
@@ -23,7 +28,7 @@ Nnsight is built on pytorch.
 Running inference on a huge remote model with nnsight is very similar to running a neural network locally on your own workstataion.  In fact, with nnsight, the same code for running experiments locally on small models can also be used on large models, just by changing a few arguments.
 
 The difference between nnsight and normal inference is that when you use nnsight, you do not treat the model as an opaque black box.
-Instead, you set up a python `with` context that enables you to get direct access to model internals while the neural network runs.
+Instead, you set up a python ``with`` context that enables you to get direct access to model internals while the neural network runs.
 Here is how it looks:
 
 .. code-block:: python
@@ -38,15 +43,15 @@ Here is how it looks:
     print('The model predicts', runner.output)
     print('The internal state was', hidden_state.value)
 
-The library is easy to use. Any HuggingFace model can be loaded into a `LanguageModel` object, as you can wee on line 2.  Notice we are loading a 70-billion parameter model, which is ordinarily pretty difficult to load on a regular workstation since it would take 140-280 gigabytes of GPU RAM just to store the parameters. 
+The library is easy to use. Any HuggingFace model can be loaded into a ``LanguageModel`` object, as you can wee on line 2.  Notice we are loading a 70-billion parameter model, which is ordinarily pretty difficult to load on a regular workstation since it would take 140-280 gigabytes of GPU RAM just to store the parameters. 
 
-The trick that lets us work with this huge model is on line 3.  We set the flag `remote=True` to indicate that we want to actually run the network on the remote service.  By default the remote service will be NDIF.  If we want to just run a smaller model quickly, we could leave it as `remote=False`.
+The trick that lets us work with this huge model is on line 3.  We set the flag ``remote=True`` to indicate that we want to actually run the network on the remote service.  By default the remote service will be NDIF.  If we want to just run a smaller model quickly, we could leave it as ``remote=False``.
 
-Then when we invoke the model on line 4, we do not just call it as a function. Instead, we use it as a `with` context manager.  The reason is that nnsight does not treat neural network models as black boxes; it provides direct access to model internals.
+Then when we invoke the model on line 4, we do not just call it as a function. Instead, we use it as a ``with`` context manager.  The reason is that nnsight does not treat neural network models as black boxes; it provides direct access to model internals.
 
 You can see what simple direct access looks like on lines 5-6.  On line 5, we grab a hidden state at layer 10, and on layer 6, we change the output of an MLP module inside the transformer at layer 11.
 
-When you run this `with`-block code on lines 5 and 6 on your local workstation, it actually creates a computation graph storing all the calculations you want to do.  When the outermost `with` block is completed, all the defined caculations are sent to the remote server and executed there.  Then when it's all done, the results can be accessed on your local workstation as shown on line 7 and 8.
+When you run this ``with``-block code on lines 5 and 6 on your local workstation, it actually creates a computation graph storing all the calculations you want to do.  When the outermost ``with`` block is completed, all the defined caculations are sent to the remote server and executed there.  Then when it's all done, the results can be accessed on your local workstation as shown on line 7 and 8.
 
 What happens behind the scenes?
 -------------------------------
@@ -58,14 +63,14 @@ An example of one such intervention graph can be seen below:
 
     An example of an intervention graph. Operations in research code create nodes in the graph which depend on module inputs and outputs as well as other nodes. Then, this intervention graph is interleaved with the normal computation graph of the chosen model, and requested inputs and outputs are injected into the intervention graph for execution. 
 
-Basic access to model internals can give you a lot of insight about what is going on inside a large model as it runs.  For example, you can use the (link needed) *logit lens* to read internal hidden states as text.  And use can use (link needed) *causal tracing* to locate the layers and components within the network that play a decisive role in making a decision.
+Basic access to model internals can give you a lot of insight about what is going on inside a large model as it runs.  For example, you can use the `logit lens <https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens>`_ to read internal hidden states as text.  And use can use `causal tracing <https://rome.baulab.info/>`_ or `path patching <https://arxiv.org/abs/2304.05969>`_ or `other circuit discovery methods <https://arxiv.org/abs/2310.10348>`_ to locate the layers and components within the network that play a decisive role in making a decision.
 
-And you can do them on large models like Llama-2-70b!
+And with nnsight, you can use thes methods on large models like Llama-2-70b.
 
-The nnsight library also provies full access to gradients and optimizations methods, out of order module applications, cross prompt interventions and much more.
+The nnsight library also provies full access to gradients and optimizations methods, out of order module applications, cross prompt interventions and many more features.
 
 See the :doc:`tutorials/basics` and :doc:`tutorials/features` pages for more information on nnsight functionality.
 
 The project is currently in Alpha pre-release and is looking for early users/and contributors!
 
-If you are interested in contributing or being an early user, please join the `NDIF Discord <hhttps://discord.gg/ZRPgsf6P>`_ for updates, feature requests, bug reports and opportunities to help with the effort.
+If you are interested in contributing or being an early user, join the `NDIF Discord <https://discord.gg/ZRPgsf6P>`_ for updates, feature requests, bug reports and opportunities to help with the effort.
diff --git a/public/about/index.html b/public/about/index.html
index 7da0a1f..c3b6091 100644
--- a/public/about/index.html
+++ b/public/about/index.html
@@ -8,7 +8,7 @@
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
 
-    <title>About nnsight: a transparent science API for black-box inference &#8212; nnsight 0.0.7 documentation</title>
+    <title>About nnsight &#8212; nnsight 0.0.7 documentation</title>
   
   
   
@@ -399,7 +399,7 @@
         <i class="fa-solid fa-home"></i>
       </a>
     </li>
-    <li class="breadcrumb-item active" aria-current="page">About...</li>
+    <li class="breadcrumb-item active" aria-current="page">About nnsight</li>
   </ul>
 </nav>
 </div>
@@ -416,15 +416,18 @@
 <div id="searchbox"></div>
                 <article class="bd-article" role="main">
                   
-  <section id="about-nnsight-a-transparent-science-api-for-black-box-inference">
-<h1>About nnsight: a transparent science API for black-box inference<a class="headerlink" href="#about-nnsight-a-transparent-science-api-for-black-box-inference" title="Link to this heading">#</a></h1>
+  <section id="about-nnsight">
+<h1>About nnsight<a class="headerlink" href="#about-nnsight" title="Link to this heading">#</a></h1>
+<section id="an-api-for-transparent-science-on-black-box-ai">
+<h2>An API for transparent science on black-box AI<a class="headerlink" href="#an-api-for-transparent-science-on-black-box-ai" title="Link to this heading">#</a></h2>
 <div class="sd-card sd-sphinx-override sd-mb-3 sd-shadow-sm docutils">
 <div class="sd-card-body docutils">
 <div class="sd-card-title sd-font-weight-bold docutils">
 How can you study the internals of a deep network that is too large for you to run?</div>
-<p class="sd-card-text">In this era of large-scale deep learning, the most interesting AI models are massive and hard to run.
-Ordinary commercial inference service APIs let you interact with huge models, but they do not let you see model internals.
-The nnsight library is different: it gives you full access to all the neural network internals.
+<p class="sd-card-text">In this era of large-scale deep learning, the most interesting AI models are massive black boxes
+that are hard to run. Ordinary commercial inference service APIs let you interact with huge
+models, but they do not let you see model internals.</p>
+<p class="sd-card-text">The nnsight library is different: it gives you full access to all the neural network internals.
 When used together with a remote service like the National Deep Inference Facility (NDIF),
 it lets you run expriments on huge open models easily, with full transparent access.
 The nnsight library is also terrific for studying smaller local models.</p>
@@ -433,15 +436,16 @@ <h1>About nnsight: a transparent science API for black-box inference<a class="he
 <figure class="align-default" id="id1">
 <img alt="../_images/remote_execution.png" src="../_images/remote_execution.png" />
 <figcaption>
-<p><span class="caption-text">An overview of the nnsight/NDIF pipeline. Researchers write simple Python code to run along with the neural network locally or remotely. Unlike with commercial inference, the experiment code can read or write any of the internal states of the neural networks being studied.  This code creates a computation graph that is sent to the remote service (if needed) and then interleaved with the execution of the neural network.</span><a class="headerlink" href="#id1" title="Link to this image">#</a></p>
+<p><span class="caption-text">An overview of the nnsight/NDIF pipeline. Researchers write simple Python code to run along with the neural network locally or remotely. Unlike commercial inference, the experiment code can read or write any of the internal states of the neural networks being studied.  This code creates a computation graph that can be sent to the remote service and interleaved with the execution of the neural network.</span><a class="headerlink" href="#id1" title="Link to this image">#</a></p>
 </figcaption>
 </figure>
+</section>
 <section id="how-you-use-nnsight">
 <h2>How you use nnsight<a class="headerlink" href="#how-you-use-nnsight" title="Link to this heading">#</a></h2>
 <p>Nnsight is built on pytorch.</p>
 <p>Running inference on a huge remote model with nnsight is very similar to running a neural network locally on your own workstataion.  In fact, with nnsight, the same code for running experiments locally on small models can also be used on large models, just by changing a few arguments.</p>
 <p>The difference between nnsight and normal inference is that when you use nnsight, you do not treat the model as an opaque black box.
-Instead, you set up a python <cite>with</cite> context that enables you to get direct access to model internals while the neural network runs.
+Instead, you set up a python <code class="docutils literal notranslate"><span class="pre">with</span></code> context that enables you to get direct access to model internals while the neural network runs.
 Here is how it looks:</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="linenos">1</span><span class="kn">from</span> <span class="nn">nnsight</span> <span class="kn">import</span> <span class="n">LanguageModel</span>
 <span class="linenos">2</span><span class="n">model</span> <span class="o">=</span> <span class="n">LanguageModel</span><span class="p">(</span><span class="s1">&#39;meta-llama/Llama-2-70b-hf&#39;</span><span class="p">)</span>
@@ -453,11 +457,11 @@ <h2>How you use nnsight<a class="headerlink" href="#how-you-use-nnsight" title="
 <span class="linenos">8</span><span class="nb">print</span><span class="p">(</span><span class="s1">&#39;The internal state was&#39;</span><span class="p">,</span> <span class="n">hidden_state</span><span class="o">.</span><span class="n">value</span><span class="p">)</span>
 </pre></div>
 </div>
-<p>The library is easy to use. Any HuggingFace model can be loaded into a <cite>LanguageModel</cite> object, as you can wee on line 2.  Notice we are loading a 70-billion parameter model, which is ordinarily pretty difficult to load on a regular workstation since it would take 140-280 gigabytes of GPU RAM just to store the parameters.</p>
-<p>The trick that lets us work with this huge model is on line 3.  We set the flag <cite>remote=True</cite> to indicate that we want to actually run the network on the remote service.  By default the remote service will be NDIF.  If we want to just run a smaller model quickly, we could leave it as <cite>remote=False</cite>.</p>
-<p>Then when we invoke the model on line 4, we do not just call it as a function. Instead, we use it as a <cite>with</cite> context manager.  The reason is that nnsight does not treat neural network models as black boxes; it provides direct access to model internals.</p>
+<p>The library is easy to use. Any HuggingFace model can be loaded into a <code class="docutils literal notranslate"><span class="pre">LanguageModel</span></code> object, as you can wee on line 2.  Notice we are loading a 70-billion parameter model, which is ordinarily pretty difficult to load on a regular workstation since it would take 140-280 gigabytes of GPU RAM just to store the parameters.</p>
+<p>The trick that lets us work with this huge model is on line 3.  We set the flag <code class="docutils literal notranslate"><span class="pre">remote=True</span></code> to indicate that we want to actually run the network on the remote service.  By default the remote service will be NDIF.  If we want to just run a smaller model quickly, we could leave it as <code class="docutils literal notranslate"><span class="pre">remote=False</span></code>.</p>
+<p>Then when we invoke the model on line 4, we do not just call it as a function. Instead, we use it as a <code class="docutils literal notranslate"><span class="pre">with</span></code> context manager.  The reason is that nnsight does not treat neural network models as black boxes; it provides direct access to model internals.</p>
 <p>You can see what simple direct access looks like on lines 5-6.  On line 5, we grab a hidden state at layer 10, and on layer 6, we change the output of an MLP module inside the transformer at layer 11.</p>
-<p>When you run this <cite>with</cite>-block code on lines 5 and 6 on your local workstation, it actually creates a computation graph storing all the calculations you want to do.  When the outermost <cite>with</cite> block is completed, all the defined caculations are sent to the remote server and executed there.  Then when it’s all done, the results can be accessed on your local workstation as shown on line 7 and 8.</p>
+<p>When you run this <code class="docutils literal notranslate"><span class="pre">with</span></code>-block code on lines 5 and 6 on your local workstation, it actually creates a computation graph storing all the calculations you want to do.  When the outermost <code class="docutils literal notranslate"><span class="pre">with</span></code> block is completed, all the defined caculations are sent to the remote server and executed there.  Then when it’s all done, the results can be accessed on your local workstation as shown on line 7 and 8.</p>
 </section>
 <section id="what-happens-behind-the-scenes">
 <h2>What happens behind the scenes?<a class="headerlink" href="#what-happens-behind-the-scenes" title="Link to this heading">#</a></h2>
@@ -469,12 +473,12 @@ <h2>What happens behind the scenes?<a class="headerlink" href="#what-happens-beh
 <p><span class="caption-text">An example of an intervention graph. Operations in research code create nodes in the graph which depend on module inputs and outputs as well as other nodes. Then, this intervention graph is interleaved with the normal computation graph of the chosen model, and requested inputs and outputs are injected into the intervention graph for execution.</span><a class="headerlink" href="#id2" title="Link to this image">#</a></p>
 </figcaption>
 </figure>
-<p>Basic access to model internals can give you a lot of insight about what is going on inside a large model as it runs.  For example, you can use the (link needed) <em>logit lens</em> to read internal hidden states as text.  And use can use (link needed) <em>causal tracing</em> to locate the layers and components within the network that play a decisive role in making a decision.</p>
-<p>And you can do them on large models like Llama-2-70b!</p>
-<p>The nnsight library also provies full access to gradients and optimizations methods, out of order module applications, cross prompt interventions and much more.</p>
+<p>Basic access to model internals can give you a lot of insight about what is going on inside a large model as it runs.  For example, you can use the <a class="reference external" href="https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens">logit lens</a> to read internal hidden states as text.  And use can use <a class="reference external" href="https://rome.baulab.info/">causal tracing</a> or <a class="reference external" href="https://arxiv.org/abs/2304.05969">path patching</a> or <a class="reference external" href="https://arxiv.org/abs/2310.10348">other circuit discovery methods</a> to locate the layers and components within the network that play a decisive role in making a decision.</p>
+<p>And with nnsight, you can use thes methods on large models like Llama-2-70b.</p>
+<p>The nnsight library also provies full access to gradients and optimizations methods, out of order module applications, cross prompt interventions and many more features.</p>
 <p>See the <a class="reference internal" href="../tutorials/basics/"><span class="doc">Basics</span></a> and <a class="reference internal" href="../tutorials/features/"><span class="doc">Features</span></a> pages for more information on nnsight functionality.</p>
 <p>The project is currently in Alpha pre-release and is looking for early users/and contributors!</p>
-<p>If you are interested in contributing or being an early user, please join the <a class="reference external" href="hhttps://discord.gg/ZRPgsf6P">NDIF Discord</a> for updates, feature requests, bug reports and opportunities to help with the effort.</p>
+<p>If you are interested in contributing or being an early user, join the <a class="reference external" href="https://discord.gg/ZRPgsf6P">NDIF Discord</a> for updates, feature requests, bug reports and opportunities to help with the effort.</p>
 </section>
 </section>
 
@@ -512,6 +516,7 @@ <h2>What happens behind the scenes?<a class="headerlink" href="#what-happens-beh
   </div>
   <nav class="bd-toc-nav page-toc">
     <ul class="visible nav section-nav flex-column">
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#an-api-for-transparent-science-on-black-box-ai">An API for transparent science on black-box AI</a></li>
 <li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#how-you-use-nnsight">How you use nnsight</a></li>
 <li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#what-happens-behind-the-scenes">What happens behind the scenes?</a></li>
 </ul>
diff --git a/public/objects.inv b/public/objects.inv
index b5f7d0cb12f0b85191cc9b77adf2e04970cf8de7..355d1cc837bda7f2e61f36170aa95b896d31b4e4 100644
GIT binary patch
delta 1980
zcmV;t2SfOQ9fKXP4+Vd281Lk7sV=a&t}NW3D$SZA<!3nMZLsRpJCwBJW1Q;D(z;4d
zWp;i^PxV`x75198A-+_}m#4>sn<s{OqRkV2xF+-D%cH6FTK`Pn>meOF<_+Cu_ecFR
z;rkN4&vqDoUb8;EiZ)+$>4{y7Do^_><3aGzLT9Zd;PN*wPIP~9qKXq<oT%b|d^#+e
z#C;1`wwUCrxoL6UE%x|OD@z}Eno1RtH%+-p*7LV~#If&M`+#b4I&V^NH{Gn5W<IN_
z*&r#w3Em+Clt%TWL?Tq7Kvp=(>ca<(X2pV;;Z*Af5gOSV2DHNoms8|vWJ?st3MYA*
zhfgDW!htQ3^2L9oe<JAu1+v0P)}NwiL~9t(ZkzCSnlE2P>s_u~@E0T0&8ry*WX>?4
z9Y(mZW`aVxfdgA&<m=TL6w);cWQCGEt^XiVJ<*_s+qA0-7uRVl*O~Z>5$Z4lB+DWe
zfDJ~hv9g6ivVj9zqUBF(XviX%h+B*?ajd>kQUtO>ZnA$yg(Ba?lSCj%5F?CU)G*t+
zpb2D!k*sFlQ7Bd<h!IAwo>WC4SffBzYsrfR4}o_Ki<J(EZOKH4zJ6t}SP}-U*$atM
zWnKLtRw)G=)lyhsg_^$@sd8GiMx=J)!OXDgjn#2f0T@WIB{mGw2D9pkd`1Mi2g=4X
z@?d6I_2z%NLaGQ%CfqbGl6j#SPW{aB7vofl4K-g0r}lpJCd`aTy;|8x9D;&wuq6vd
z&Z;JlMpCyJQ^nJh$I?UN=*eP`3uejVkkl>4RPm@e3)Jx_>K5a;c+?^qy)+mX#A20k
z&|)oDFZc-zw)JA3WVuYp&03F&_2T1j4Gu?ey{mt9OqpJ~r|!&)Er^&6tCr!WB<l65
zX3`imbE^YUJb_32m~hNTeV{!}FpKruv_UP<k$yBmPHV_%1DvoCKD2?c{DVT{;3zMe
z@W$C&w6TrU?JhJ?O^f$w!<w)WJ~V;p+yS&<Y3ODTnvlXTbR$Aqz1jtP(1c{J>86d#
zB*%Zb(L`vhDyI$507rQdg*PUQAP#QCM)=SKX3JMW8>6iToI5=7T&JfP@7y}?#r6Ol
z2{b@fSh=xRl`m5V+!U_9)OjFumuGH8sP|>_fSZ!2AG3Xs1~Ec6d(ec0yCXsKm&Ccz
zM2Na00TZZ1dC`P7=UJhRZsx`~Sz;4QLc)LVk3`0aWSOqQ&Ds_DuK=Dj<Nv&%2N>vq
zgC0Df`xjGUAq@a+cLX1aRfZ3Ez4pX8*1cP+U+2(?RkP+mjN-M1CpiGcuBJG6tv1U=
z9#P+cj5~g2LLr>2<MIoR(;`x$%)f(FC>bGx&HBvrN6`XJNYG-lMC`@ky(S_rg++gk
zCZsZN+cQmKRpMHo2?<(kmKdIu>sPm$y>BbrtVynlj8-JQ-9>9PQ^ujdIAzK)l&o_V
z`H1UV{FF-zQj^nplY+a{O2Jq2vj9g5FZ=EY6*8R{LW%Tu7sV(V7nveNX@s{lP8#yW
z?HkYO+c&W%(ac|lQD<_(CUHttZZv<KO2zOON2v36!y#QpC%i6*(jwE0q4Y?CuP{#6
zm=O-AZKQ+@)*8pO4S~DGn57|ErF-z;*1i3hz79}}%XLHL>$DUXm3}R>9$-@7OL^xh
zi<A^<oDf-y=IJPpmS@SWhn8z$ZnTdLXUvO*4PoQ}Y+|Lt7vq2wI~L-EO=Exc&$$V)
z(q|ILDpv3!{~=cM0tPjVRX(Lz#HyY^5Tj_dGK(Wxs-y%hVnx!lg7E1_op&?!B#Bhb
z0@}sO7Wpl)as?w$9xG4f&BO{+l%Ped$SEf$R_6qQ7{#g$y$1m)UAzu)8EjEl$uxu_
ziCT=bIghBbBudOGttd*`iUohPi<2Fi?gUV~nA~J9j?;!2_T48{4O+yBT%=mXDP5po
zX0e)8Hdm}%MG0EOi5xS<;zW*=phcX>ISVaL=F9=u#7T{RmR@Jd4V1^p3(vd+=v&RS
zbQ#1coHO#`M9v(5O`KGj?iZ(0BK*a1+CtO(0NNJw`0T}T+Ez)4aVmdTFsLC;`6B%>
zjs694n|W+3WFslyKng#OG7~#fNM5e?^IF9To-;?|<jx#`O`Oy@Gc``?%mLWMN!6LJ
zaXK|5WDuv2;y{KD;^qU`%t@%Dq?sAIJ!O%S;-5C}@_vU5Vihhjfn${}U{J$I<x|e_
zc+pD^z-BE~ros+<P?J}QuoE+Rwu`d<7$$S0hsahZP;Hhh=E2G=KKHv|HCoc2Z>!z{
z2$aKw85z~L2{IfV`(Xl9Qub|iE1&yauo^?}>ns&6-p);X;MFy9iAZ4Duch8)&VQ5E
z3nl{9zLWk7;{gkk;tbS(asJVr`@gHAt-XJs#hy#m?e6&lD(?JHZX6(2s%=@J8|cUf
zWt+k8h1eDa+GlyXF`ha5)+byy$vL0%eq8H@Se{`oI@Q;*^^SWFstJCYO*D6EbA>U4
zvyStAS&s7ffwa`uyr*NqA<@qSs%E3a<MQ<0Uqx49dUh;p*R$12)xJaWmnwV8F6@)k
zX~yOlWiQ8<+SWQSU61zc4y(cB{gBM=W*PfgZo+n7>+mdeXS1WW-;l$1iBL`OTU@yJ
O4b=s@Vf-KIt`Ets0ng<C

delta 1978
zcmV;r2SxaU9f2LN4+VezImr^<PO477=jix2q~aaum#4>spP}SbX6KjmRKF!f^~_pP
zX7G3Nw^XyzEHzN6Mx|L(q#Tt_sB}Wv^r4xQ@m=em>3cn-L&ru%R?RlMKkA<e-<R-x
zqZ$+a^P2VPRa_BOm!8<QsPeSGG9DryOLW#+0xo~^;zSoGsyKh)#fd8Z$EU-h!Q8ix
zWqV4#nwu8q-C~aqwX*bqr-@b}dDDcfWIemfM;v>$wGXH!r}HKScazbIX<oFNxDApL
zoc<j$KxtIZQ6xeY3S@<otUim-XjUwk8BVo+OrepjVL&^aa5;gVMz%zOtZ<U2S^6}x
zCmh%kDPJrJAd-JBP#`OuWc{IvMzn?j?Y0STr`hyXwBF^q27fU^-MrR;K;{es+F^tn
zD=H|Y8#u5fM!sIVK_OkEKvpQp)9Mfs)e{YBxJ|pdaB-c+autfd7@-a`K(Z`x0oY)~
z8tYytBpW!eC0hQp!iFq@iMYiW6UXWcCPg4C<R)uWDDr<jJxK(T1Tn(sMGdp9%bh@0
z7|CjG9))5>f*4`+>M2(gf;9?cwU)eC4iR|AuviC?*cMcT=<8Poiv?rQn!S)HRo1m3
zVhvNUQ7wf9*01@Ckt(M(ZA5A(9?T4@-dH<F6@Y;RTVlf?Z7{2z$Y(^Ld!TGQBM)YV
zRd233q>6vQWWr73BAJ(=;ndF@e=$y_*if^XaBA<@cEZev)T?!$#33l?23xXV<eY2r
zXe4!uF;zT0c`Q9Nj-D(Axtx|f4oTf&Ocjrs(Lf!KqHZyci$^W7(F=!hK`hoF2QAie
z^>U!VU|TQsNtTO<+^qGeST9Wu*Whpj*SlKfl<9wked^A<*n)`Ju%;PqN}^t`c_xiP
zGq*Yr#S?hMj|s<o)Cbzr1hZJ(O&in#9qC6C<g~(^HoyrR;X@l3%ReYI4vzAo32&U+
zMH|~l-R?pY)wHyqHmnI7;X@Oc&MZJ1mWFQjpb07bLN_9$)vH~w2Te%kif`JuOmdtX
zO@x2OnseF!4RDkfQFvoI3F6>JY=jR@V76=)v@zOhz`4UC&vo*O@y@NYVr&o4kzNC2
zg_Rq7Rr&IGz)j)mOPwV`cX{SkgnD1D54b6b`Y{&>X%HiHvj<H`xH}RwyGfiIO@ydB
z5-`0=low5Sa~2lb=w@z=lO;B>U?lwhNaTN`NS4Vg+^k)Z{|ewqGycyDdVql*IOxFx
zx_>bxme~N%c1Q4$SmpnK*K1F!W8J&8`gMk#ST$=7#3){Cc#;E9>}raG*J`s|<Pr59
z$hhNYrW?Y^IxfH9I4vS2%IrKyg_036*sRY?jub7>gaj=%OT=Cr-fJTAQdnegLMngr
zwms7%Rwb_WnUJ8xW{Kfhxqfx4+55J_&6?zz$!JB=+g-F)Gvy-+j8mozM9Dgnk&n2(
z#ZQ^GAT>FiHz~MFycB#jKMQaq__FVgP$83jA(TjecTtR@alt7<lty?<<D?-^+`jRg
zzI_vW63y&p7<DEmY!atbWlqDXR1ANAafCX5HyqMsbi(U`C@nGx8A^{N_zL56jrrnm
z+D1yqV6AaX>JYeFj5!>VRWb+<Zr$6D$?X8OxLh|>zD`<kQOVgt>j5SOzJz$5vPemx
z#tD(NXr7MpXnB^*duX{9=0^M2aK@}z*bqhzz$R8Id@&A4v11`l*fduEoVkAxD}5$`
ztYQT(vLj+OFJMr^Smjd^My%=y1Tl(MD|0-erAkWBB32|#D+r%{)Oj}(Q<6y4ETCPi
zY?1vED_1ZA<+1WqR!yuxMG0EOikvcbVs%a+h*7NC(0dS&(#7i#m%$c=l}u77Qmn;D
zo3o5MOQOWAl8&OJtyn<2IN5)p=}rK(i^)y);y7)XVc&hq)u2V3$VDPnoYDmfW)`bi
z<$A@+Rg|DboX9bcEKcM|30lO7oO9ITWX>FbO`O#DXX$kw-9UMqyztCRfWFmCOP4{M
z!Z}|rPUOr1*u+Vd$$)VxCBk1Er!6$i51?%^kI!Blr)`z87^iXtgBpM0lrNGa)97Cy
zx0%PrLN<~D4y5qoC^PLdh2-UGKd)7s;5n}}PVUSB*u+Vl^H$@e&K!VEoK&3`8>dr4
zLI!aPDGp@lAZ|W@&77h-N}8F!+fx=PDgJ5mE-QG*AXecbPdHZT0tPjVR6b=Sj~Bh<
z0BqJ$WuolB2Q`@{J26yKYr81xk6|)%dWdXw0@Y^8aUQJ9;&Z<XR-+|3`nKvVfIvA+
zn2~RNn;^r{u^%QtrEK3;xAM8)1*<XizRptN;_ckD2VPwxmxu(W{aWf>{zU+j*$XBD
z5x<iG4C4X&ljRK5f1&)7JokT!MOWbcn=bbBt8RDC-&}F0lyc($=~Znj5#2yX3Mt#9
ze=o$gj?g|a)Q#~>^S3?}yGhO|p!Y*zH^lPTd(qj!mJNQ~dr+<R(`=%-)9AJ^Mt#<y
z;V;X9AwQ6o`kME2EI1_kF+$aBlz3d8-utWQeN4}eW$k)`OuO25Nd8h~PuYcik~+=U
z90T*^kW|}R$Fl1IqTOLN{=6T>+1)I|MazBL?rR;<g>Hs+)b@LL_{I{d34T`$ci5r2
MK=+vc1DEhF_Ys4=B>(^b

diff --git a/public/searchindex.js b/public/searchindex.js
index 608f394..a9aa5d5 100644
--- a/public/searchindex.js
+++ b/public/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["about", "documentation", "documentation/contexts", "documentation/intervention", "documentation/models", "documentation/module", "documentation/patching", "documentation/tracing", "documentation/util", "index", "start", "tutorials", "tutorials/basics", "tutorials/features", "tutorials/main_demo", "tutorials/notebooks/cross_prompt", "tutorials/notebooks/multiple_token", "tutorials/notebooks/token_indexing"], "filenames": ["about.rst", "documentation.rst", "documentation/contexts.rst", "documentation/intervention.rst", "documentation/models.rst", "documentation/module.rst", "documentation/patching.rst", "documentation/tracing.rst", "documentation/util.rst", "index.rst", "start.rst", "tutorials.rst", "tutorials/basics.rst", "tutorials/features.rst", "tutorials/main_demo.ipynb", "tutorials/notebooks/cross_prompt.ipynb", "tutorials/notebooks/multiple_token.ipynb", "tutorials/notebooks/token_indexing.ipynb"], "titles": ["About nnsight: a transparent science API for black-box inference", "Documentation", "nnsight.contexts", "nnsight.intervention", "nnsight.models", "nnsight.module", "nnsight.patching", "nnsight.tracing", "nnsight.util", "nnsight", "Getting Started", "Tutorials", "Basics", "Features", "Main Demo", "Cross-Prompt Intervention", "Multiple Token Generation", "Token Based Indexing"], "terms": {"can": [0, 2, 3, 4, 5, 7, 11, 14, 15], "studi": 0, "intern": [0, 9, 10, 14], "deep": [0, 10], "network": [0, 9], "i": [0, 2, 3, 4, 5, 7, 8, 9, 10, 14, 15, 16, 17], "too": 0, "larg": [0, 9], "run": [0, 2, 4, 5, 7, 8, 12], "In": [0, 2, 4, 14, 15, 17], "thi": [0, 2, 3, 4, 5, 7, 8, 10, 14, 15, 17], "era": 0, "scale": 0, "learn": [0, 10, 11], "most": [0, 2, 5], "interest": 0, "ai": 0, "model": [0, 1, 2, 3, 5, 7, 9, 10, 12, 15, 16, 17], "ar": [0, 2, 3, 4, 5, 7, 8, 14, 17], "massiv": 0, "hard": 0, "ordinari": 0, "commerci": 0, "servic": 0, "let": [0, 14, 17], "interact": [0, 3, 4, 7], "huge": 0, "thei": [0, 2, 7], "do": [0, 2, 3, 7], "see": [0, 4, 11, 14, 17], "The": [0, 2, 3, 4, 5, 6, 7, 14, 16, 17], "librari": 0, "differ": [0, 2, 17], "give": [0, 4, 8, 14], "full": 0, "access": [0, 2, 3, 4, 5, 9], "all": [0, 2, 3, 4, 5, 7, 8], "neural": [0, 9], "when": [0, 2, 3, 4, 7, 16, 17], "togeth": [0, 2], "remot": [0, 2], "like": [0, 2, 4, 9, 10, 14], "nation": 0, "facil": 0, "ndif": [0, 2], "expriment": 0, "open": 0, "easili": [0, 3], "also": [0, 2, 4, 5, 6, 11, 14, 15], "terrif": 0, "smaller": 0, "local": [0, 4], "an": [0, 2, 3, 4, 5, 6, 7, 8, 14], "overview": 0, "pipelin": 0, "research": 0, "write": 0, "simpl": [0, 4, 8], "python": [0, 7, 17], "code": [0, 9], "along": [0, 3], "unlik": 0, "experi": [0, 10], "read": 0, "ani": [0, 2, 3, 4, 6, 7, 8, 9, 14], "state": [0, 3, 4, 14, 16, 17], "being": [0, 7, 14], "creat": [0, 2, 4, 5, 7, 14], "comput": [0, 2, 3, 4, 7, 8, 14], "graph": [0, 2, 3, 4, 5, 7], "sent": 0, "need": [0, 2, 4, 7, 8], "interleav": [0, 3, 4, 7], "execut": [0, 2, 3, 4, 5, 7, 8, 14], "built": [0, 2, 7], "pytorch": [0, 4], "veri": 0, "similar": 0, "your": [0, 14], "own": [0, 7], "workstataion": 0, "fact": 0, "same": [0, 2, 7, 8, 15], "small": 0, "just": [0, 7, 17], "chang": [0, 14], "few": [0, 14], "argument": [0, 2, 3, 4, 7, 8], "between": [0, 15], "normal": [0, 4, 17], "treat": [0, 14], "opaqu": 0, "instead": [0, 8], "set": [0, 2, 3, 5, 7, 9], "up": [0, 9], "context": [0, 1, 3, 4, 5, 6, 9, 14], "enabl": [0, 3, 4, 9], "get": [0, 2, 5, 7, 8, 9, 14, 15, 16, 17], "direct": [0, 4, 7, 14], "while": [0, 8, 16], "here": [0, 2, 4, 14, 15, 16], "look": 0, "from": [0, 2, 3, 4, 6, 7, 8, 9, 14, 15, 16, 17], "import": [0, 4, 6, 11, 14, 15, 16, 17], "languagemodel": [0, 2, 4, 14, 15, 16, 17], "meta": [0, 2, 4, 5, 7], "llama": 0, "2": [0, 14, 15, 16, 17], "70b": 0, "hf": 0, "forward": [0, 2, 4, 5, 7, 8, 12, 17], "true": [0, 2, 4, 7, 8, 14], "runner": [0, 2, 3, 4, 14, 17], "invok": [0, 2, 3, 4, 5, 14, 15, 16, 17], "eiffel": [0, 2, 3, 4, 5, 16, 17], "tower": [0, 2, 3, 4, 5, 16, 17], "citi": [0, 2, 3, 4, 5, 14, 15, 16, 17], "hidden_st": [0, 2, 3, 5, 17], "layer": [0, 14, 16], "10": [0, 14, 16], "input": [0, 2, 3, 4, 5, 8, 14, 17], "save": [0, 2, 3, 4, 5, 14, 15, 16, 17], "one": [0, 4, 5, 7, 8, 14, 16], "hidden": [0, 3, 14, 16, 17], "11": [0, 4, 14], "mlp": [0, 3, 4, 5, 14], "output": [0, 2, 3, 4, 5, 7, 14, 15, 16, 17], "0": [0, 2, 3, 4, 5, 7, 14, 15, 16, 17], "modul": [0, 1, 2, 3, 4, 6, 7, 8, 9, 14], "print": [0, 2, 3, 4, 5, 8, 14, 15, 16, 17], "predict": [0, 4], "wa": [0, 4, 14], "valu": [0, 2, 3, 4, 5, 7, 14, 15, 16, 17], "easi": [0, 2], "huggingfac": [0, 4, 9, 14], "load": [0, 4, 12], "object": [0, 2, 3, 4, 5, 6, 7, 8], "wee": 0, "line": [0, 9, 14], "notic": [0, 17], "we": [0, 4, 5, 7, 14, 15, 16, 17], "70": 0, "billion": 0, "paramet": [0, 2, 3, 4, 5, 6, 7, 8, 14], "which": [0, 2, 4, 5, 7, 8, 14], "ordinarili": 0, "pretti": 0, "difficult": 0, "regular": 0, "workstat": 0, "sinc": [0, 8, 14], "would": [0, 3, 14], "take": [0, 3, 8], "140": 0, "280": 0, "gigabyt": 0, "gpu": [0, 4, 14], "ram": 0, "store": [0, 3, 7, 14], "trick": 0, "u": 0, "work": [0, 2, 3, 14, 15], "3": [0, 3, 4, 14, 15, 16, 17], "flag": 0, "indic": [0, 3, 14], "want": [0, 2, 14], "actual": [0, 2, 3, 4, 7, 14], "By": 0, "default": [0, 2, 4, 7, 8], "If": [0, 2, 3, 4, 7, 8, 10, 14], "quickli": 0, "could": [0, 3, 15], "leav": 0, "fals": [0, 2, 4, 8, 14], "Then": 0, "4": [0, 14, 17], "call": [0, 2, 3, 4, 5, 6, 7, 8, 14, 15, 16, 17], "function": [0, 2, 3, 4, 5, 6, 7, 8, 11, 17], "manag": [0, 2, 3, 4, 6], "reason": 0, "doe": [0, 5, 14], "provid": [0, 2, 3, 4], "5": [0, 14], "6": [0, 14], "On": [0, 2], "grab": [0, 15], "insid": [0, 14], "transform": [0, 2, 3, 4, 5, 8, 9, 15, 16, 17], "block": [0, 2, 9, 15], "calcul": 0, "outermost": 0, "complet": [0, 2, 3, 7, 14], "defin": [0, 8], "cacul": 0, "server": [0, 2], "": [0, 2, 3, 4, 5, 7, 8, 14, 17], "done": [0, 3], "result": [0, 2, 3, 5, 7, 14, 17], "shown": [0, 15], "7": [0, 14], "8": [0, 14], "help": 0, "understand": 0, "oper": [0, 2, 4, 5, 7, 14, 15], "immedi": 0, "add": [0, 2, 3, 4, 5, 6, 7], "intervent": [0, 1, 2, 4, 5, 9, 13, 16], "alongsid": 0, "upon": [0, 3], "exit": [0, 2, 3, 14], "exampl": [0, 2, 3, 4, 5, 10, 12, 17], "seen": 0, "below": [0, 2, 5, 11], "node": [0, 3, 5, 7], "depend": [0, 7], "well": [0, 2, 7], "other": [0, 4, 7], "chosen": [0, 2], "request": [0, 5], "inject": 0, "basic": [0, 10, 11], "lot": 0, "insight": 0, "go": [0, 4], "For": [0, 3, 14], "link": 0, "logit": [0, 2, 3, 4, 5, 8, 12], "len": [0, 12], "text": [0, 14, 15, 16, 17], "And": [0, 6], "causal": [0, 14], "trace": [0, 1, 2, 3, 4], "locat": [0, 14, 15], "compon": [0, 14], "within": [0, 2, 4, 5, 8, 14, 15], "plai": 0, "decis": 0, "role": 0, "make": [0, 2, 3, 14], "them": [0, 2, 3, 4, 5, 7, 8, 15], "provi": 0, "gradient": [0, 9], "optim": 0, "method": [0, 2, 3, 4, 7, 8, 14, 15, 16, 17], "out": [0, 2, 4, 6, 7, 10, 11, 14], "order": [0, 2, 3, 4, 5, 7], "applic": 0, "cross": [0, 8, 13], "prompt": [0, 2, 4, 13, 14, 16], "much": 0, "more": [0, 4, 7, 10, 11, 14, 16], "featur": [0, 2, 8, 9, 10, 11], "page": 0, "inform": [0, 2, 4, 7, 14], "project": 0, "current": [0, 2, 3, 7], "alpha": 0, "pre": [0, 2, 7, 15], "releas": 0, "earli": 0, "user": [0, 3, 4, 5], "contributor": 0, "contribut": 0, "pleas": [0, 14, 15, 16, 17], "join": 0, "discord": 0, "updat": [0, 2, 3, 7, 8], "bug": 0, "report": 0, "opportun": 0, "effort": 0, "nnsight": [1, 10, 11, 14, 15, 16, 17], "util": 1, "patch": 1, "contain": [2, 3, 4], "logic": [2, 3], "primari": [2, 4], "two": [2, 4, 15, 17], "class": [2, 3, 4, 5, 6, 7, 8], "involv": [2, 3, 7], "tracer": [2, 5], "around": [2, 3, 4], "meta_model": [2, 4], "abstractmodel": [2, 4, 5], "track": [2, 4, 7, 14], "perform": [2, 4, 5, 7, 8, 14, 17], "said": [2, 7], "expos": 2, "attribut": [2, 3, 4, 5, 6, 7, 8], "To": [2, 10, 14], "know": 2, "about": [2, 14], "so": [2, 3, 7, 14, 17], "each": [2, 7, 14, 16, 17], "keep": 2, "batch_siz": [2, 3], "recent": [2, 5], "gener": [2, 3, 4, 5, 7, 13, 15], "index": [2, 3, 7, 13, 14], "multi": [2, 4], "iter": [2, 3, 4], "made": 2, "dure": [2, 3, 5, 7, 12], "its": [2, 4, 7, 9], "batched_input": 2, "ad": [2, 3, 4, 7], "should": [2, 3, 7, 8, 16], "format": 2, "where": [2, 4, 7], "batch": [2, 3, 4, 8], "allow": [2, 3, 4, 7], "thing": 2, "consist": [2, 14], "valid": [2, 7], "both": [2, 3, 4, 17], "becom": [2, 14], "case": [2, 4, 14, 15], "regardless": 2, "whether": 2, "string": [2, 5, 7, 8, 17], "process": [2, 7], "dictionari": 2, "id": [2, 4, 8, 9, 14], "onli": [2, 3, 4, 5, 7, 8, 9, 14], "us": [2, 3, 4, 7, 8, 14, 15, 16, 17], "carri": [2, 6, 7], "what": [2, 7], "accept": [2, 4], "parent": [2, 6, 7], "appropri": 2, "respect": [2, 5], "enter": [2, 4, 15], "some": [2, 7, 8, 10, 11, 14], "leverag": [2, 4], "prepar": [2, 4, 7], "version": [2, 4, 7], "size": [2, 3, 5, 8, 14, 16], "It": [2, 4, 7], "through": [2, 3, 4, 5, 7, 8, 11, 14], "dtype": [2, 5], "base": [2, 3, 7, 13], "characterist": 2, "come": [2, 11, 15], "implement": [2, 4, 7], "_gener": [2, 4], "_forward": [2, 4], "rich": [2, 17], "new": [2, 7, 15], "eventu": 2, "ha": [2, 4, 7, 8], "assum": 2, "arg": [2, 3, 4, 7, 8], "bool": [2, 4, 7, 8], "kwarg": [2, 4, 7, 8], "given": [2, 7, 8, 14], "_run_loc": 2, "show": [2, 5, 14], "gpt2": [2, 4, 5, 15, 16, 17], "after": [2, 4, 8, 14, 16], "lm_head": [2, 3, 4, 5], "zero": [2, 5, 14], "h": [2, 3, 4, 5, 8, 16, 17], "otherwis": [2, 5, 7], "type": [2, 3, 4, 5, 6, 7, 8], "option": [2, 4, 7, 8], "hang": 2, "until": [2, 14], "job": 2, "return": [2, 3, 4, 5, 7, 8, 14, 16], "you": [2, 3, 4, 7, 10, 11, 14, 15, 16, 17], "retriev": [2, 3, 8], "run_loc": 2, "local_model": [2, 4], "th": 2, "later": [2, 14], "posit": [2, 3, 4, 7, 14], "pass": [2, 4, 5, 7, 8, 9, 12], "list": [2, 3, 4, 5, 6, 7, 14], "keyword": [2, 4, 7], "dict": [2, 4, 7, 14], "str": [2, 3, 4, 5, 7, 8], "proxi": [2, 3, 5, 7, 14], "int": [2, 4, 7], "generation_idx": [2, 7], "idx": [2, 3, 17], "__exit__": [2, 3, 6], "scan": 2, "meant": [2, 4], "tandem": [2, 4], "initi": [2, 4, 5, 16], "post": [2, 7], "_prepare_input": 2, "_scan": 2, "potenti": [2, 3], "correctli": [2, 14, 17], "free": [2, 7], "wise": 2, "mai": 2, "turn": 2, "loop": 2, "shape": [2, 3, 5, 7, 8, 14, 16], "error": [2, 7], "have": [2, 3, 7, 15, 17], "first": [2, 3, 4, 5, 7, 8, 14, 15, 17], "next": [2, 16], "increment": [2, 7], "1": [2, 4, 8, 14, 15, 16, 17], "none": [2, 3, 4, 6, 7], "design": [2, 3], "subsequ": [2, 3, 16], "appli": [2, 3, 4, 8, 10, 11, 14, 16], "how": [2, 5, 10, 11, 14, 16, 17], "mani": [2, 7], "onc": 2, "save_al": 2, "module_path": [2, 3, 5], "kei": [2, 3, 4, 7, 9, 11], "interventionproxi": [3, 5], "extend": 3, "fx": 3, "easier": 3, "interven": [3, 12], "entri": [3, 7], "hook": [3, 4, 5, 7, 8], "hookmodel": 3, "remov": [3, 7], "module_kei": 3, "input_hook": 3, "callabl": [3, 7, 8], "output_hook": 3, "backward_input_hook": 3, "backward_output_hook": 3, "regist": [3, 8], "__enter__": [3, 6], "root": [3, 4, 5, 7, 8], "torch": [3, 4, 5, 7, 8, 14, 16], "nn": [3, 4, 5, 7], "tupl": [3, 7, 14], "signatur": 3, "handl": [3, 6], "removablehandl": 3, "sub": [3, 4], "addit": 3, "delet": 3, "listen": [3, 7], "had": 3, "been": [3, 8], "token": [3, 4, 8, 13, 14, 15], "t": [3, 14, 17], "multipl": [3, 7, 13, 17], "pad": [3, 14, 15, 16, 17], "mean": [3, 14], "shift": [3, 8], "care": [3, 8], "collect": [3, 7, 8], "tensor": [3, 4, 5, 7, 8, 15, 17], "popul": [3, 5, 7], "properti": [3, 5, 7], "union": [3, 4, 7], "tokenindex": 3, "alia": 3, "directli": [3, 4], "second": [3, 15], "dimens": 3, "neg": 3, "left": [3, 17], "equival": 3, "helper": [3, 8, 17], "aid": 3, "activ": [3, 4, 9, 14], "form": [3, 4, 7], "path": [3, 4, 5, 8], "check": [3, 4, 7, 10, 11], "argument_node_nam": [3, 7], "exist": [3, 5], "name": [3, 4, 7], "batch_start": 3, "narrow": 3, "select": 3, "relev": [3, 14], "view": [3, 7, 14], "oppos": 3, "copi": 3, "downstream": [3, 4, 5], "edit": 3, "origin": [3, 6, 14], "prevent": 3, "effect": 3, "bath": 3, "outsid": [3, 7], "preview": 3, "either": [3, 7], "rel": [3, 5], "denot": [3, 5, 16], "modifi": 3, "main": [4, 11, 12], "wrap": [4, 5, 7, 8], "repo": [4, 9, 14], "device_map": [4, 14, 15, 16, 17], "cuda": [4, 14, 15, 16, 17], "declar": 4, "entail": 4, "underli": [4, 14], "automodelforcausallm": 4, "unus": 4, "acceler": 4, "packag": [4, 8, 9, 10, 11], "encompass": 4, "displai": 4, "structur": [4, 14], "abil": 4, "itself": 4, "gpt2lmheadmodel": 4, "gpt2model": 4, "wte": [4, 15], "embed": [4, 14, 15], "50257": 4, "768": [4, 16], "wpe": 4, "1024": 4, "drop": 4, "dropout": [4, 14], "p": [4, 14], "inplac": [4, 14], "modulelist": [4, 5, 14], "12": [4, 14], "x": [4, 5, 8, 14], "gpt2block": 4, "ln_1": 4, "layernorm": [4, 14], "ep": [4, 14], "1e": [4, 14], "05": [4, 14], "elementwise_affin": [4, 14], "attn": 4, "gpt2attentionalt": 4, "c_attn": 4, "conv1d": 4, "c_proj": 4, "attn_dropout": 4, "resid_dropout": 4, "queri": 4, "wrappermodul": [4, 8], "ln_2": 4, "gpt2mlp": 4, "c_fc": 4, "act": [4, 5, 7, 14], "newgeluactiv": 4, "ln_f": 4, "linear": [4, 14], "in_featur": [4, 14], "out_featur": [4, 14], "bia": [4, 14], "determin": 4, "behavior": 4, "three": [4, 16], "invoc": [4, 15, 17], "languag": [4, 14], "over": [4, 14], "max_new_token": [4, 15, 16], "logits1": 4, "empir": 4, "build": 4, "logits2": 4, "infer": 4, "repoid_path_model": 4, "dispatch": 4, "alter": 4, "abstract": 4, "wish": [4, 5], "gain": 4, "repoid_path_clsnam": 4, "hug": 4, "face": 4, "checkpoint": 4, "custom": 4, "bee": 4, "yet": [4, 14], "init": 4, "custom_model": 4, "devic": [4, 7, 17], "ran": [4, 7], "specif": [4, 5, 14, 17], "ultim": 4, "A": [4, 7], "doc": [4, 9], "limit": 4, "amount": 4, "anf": 4, "extra": 4, "fwd_arg": 4, "rtype": 4, "node_nam": 4, "module_nam": 4, "_summary_": 4, "_description_": [4, 7], "wrapper": [4, 8], "config": 4, "file": 4, "repositori": [4, 10], "pretrainedconfig": 4, "lm": 4, "pretrainedtoken": 4, "pretrainedmodel": 4, "diffusermodel": 4, "diffus": 4, "entrypoint": 5, "interwoven": 5, "manipul": [5, 9, 10], "requir": 5, "correct": [5, 7, 14, 17], "tree": [5, 14], "repres": [5, 6, 7], "separ": [5, 8], "e": [5, 7, 8], "reset": [5, 7], "output_shap": 5, "nest": 5, "output_typ": 5, "input_shap": 5, "input_typ": 5, "must": [5, 7], "manual": 5, "backward_input": 5, "therefor": [5, 7], "time": [5, 8, 14], "refer": [5, 7], "alreadi": [5, 14], "backward_output": 5, "static": [5, 7], "our": [5, 14], "data": [5, 7, 8, 14], "obj": 6, "replac": [6, 7, 15], "importlib": 6, "import_modul": 6, "self": 6, "__module__": 6, "setattr": 6, "__name__": 6, "restor": [6, 14], "patcher": 6, "point": [7, 9], "flow": 7, "AND": 7, "proxy_valu": 7, "datatyp": 7, "set_valu": 7, "bi": 7, "convert": 7, "respons": 7, "similarli": 7, "ceas": 7, "destroi": 7, "memori": 7, "re": [7, 14, 15, 16, 17], "remain": 7, "traceabl": 7, "magic": 7, "__torch_function__": 7, "find": 7, "shame": 7, "proxy_class": 7, "reserv": 7, "target": [7, 8], "There": 7, "singl": 7, "__init__": [7, 8], "compil": 7, "whatev": 7, "Their": 7, "map": 7, "These": [7, 14], "kick": 7, "rtn": 7, "null": 7, "never": 7, "possibl": 7, "etc": 7, "name_idx": 7, "target_nam": 7, "number": 7, "previou": 7, "uniqu": 7, "module_proxi": 7, "pull": 7, "rais": 7, "valueerror": 7, "than": [7, 14, 15, 16, 17], "final": [7, 14], "module_0": 7, "consid": [7, 14], "specifi": [7, 14], "_type_": 7, "desc": 7, "action": 7, "whole": 7, "_proxy_devic": 7, "remaining_listen": 7, "remaining_depend": 7, "log": 7, "destruct": 7, "noth": 7, "fulfil": 7, "prepare_input": 7, "move": [7, 14], "prepare_proxy_valu": 7, "proxy_devic": 7, "lazi": 7, "creation": 7, "redund": 7, "event": 7, "now": [7, 14, 17], "proxy_upd": 7, "value1": 7, "value2": 7, "proxy_wrapp": 7, "fn": [7, 8], "problemat": 7, "sometim": 7, "throughout": 8, "element": 8, "everi": 8, "overridden": 8, "subclass": 8, "although": 8, "recip": 8, "instanc": 8, "afterward": 8, "former": 8, "latter": 8, "silent": 8, "ignor": 8, "cl": 8, "member": 8, "kind": 8, "cross_entropy_loss": 8, "target_id": 8, "avg_batch": 8, "avg_token": 8, "entropi": 8, "loss": 8, "n": 8, "last": [8, 16], "averag": 8, "across": [8, 16], "fetch_attr": 8, "hierarchi": 8, "level": 8, "fetch": 8, "func": 8, "lggr": 8, "decor": 8, "__class__": 8, "interpret": [9, 10, 11], "\u025bn": [9, 10], "sa\u026at": [9, 10], "start": [9, 14], "tutori": [9, 10], "integr": 9, "weight": [9, 14], "intuit": 9, "coupl": 9, "interoper": 9, "grad": 9, "train": 9, "lora": 9, "probe": 9, "pip": 10, "section": 10, "colab": 10, "d": 10, "hand": [10, 14], "cover": 10, "demo": [11, 12], "quick": 11, "walkthrough": 11, "walk": 11, "advanc": 11, "toolkit": 11, "familiar": 11, "yourself": 11, "tool": 11, "soon": 11, "paper": 11, "pretrain": 14, "ll": 14, "eleutherai": 14, "1b": 14, "pythia": 14, "onto": 14, "abov": 14, "command": 14, "download": 14, "configur": 14, "machin": 14, "even": 14, "though": 14, "still": 14, "inspect": 14, "gptneoxforcausallm": 14, "gpt_neox": 14, "gptneoxmodel": 14, "embed_in": 14, "50304": 14, "2048": 14, "emb_dropout": 14, "15": 14, "16": 14, "gptneoxlay": 14, "input_layernorm": 14, "post_attention_layernorm": 14, "post_attention_dropout": 14, "post_mlp_dropout": 14, "attent": 14, "gptneoxattent": 14, "rotary_emb": 14, "gptneoxrotaryembed": 14, "query_key_valu": 14, "6144": 14, "dens": 14, "attention_dropout": 14, "gptneoxmlp": 14, "dense_h_to_4h": 14, "8192": 14, "dense_4h_to_h": 14, "geluactiv": 14, "final_layer_norm": 14, "embed_out": 14, "extract": 14, "residu": 14, "stream": 14, "john": 14, "mari": 14, "went": 14, "milk": 14, "last_hidden_st": 14, "aren": 14, "39": [14, 15, 16, 17], "gptneoxtokenizerfast": 14, "note": [14, 15, 16, 17], "fast": [14, 15, 16, 17], "__call__": [14, 15, 16, 17], "faster": [14, 15, 16, 17], "encod": [14, 15, 16, 17], "follow": [14, 15, 16, 17], "13": 14, "break": 14, "down": 14, "tell": 14, "modif": 14, "detail": 14, "cach": 14, "desir": 14, "As": [14, 17], "probabl": 14, "One": 14, "techniqu": 14, "decod": [14, 15], "intermedi": 14, "crude": 14, "approxim": 14, "def": 14, "norm": 14, "unembed": 14, "all_prob": 14, "prob": 14, "softmax": 14, "dim": 14, "append": 14, "distribut": 14, "evolv": 14, "plotli": 14, "express": 14, "px": 14, "mary_token": 14, "input_id": 14, "layer_prob": 14, "item": 14, "titl": 14, "accord": 14, "label": 14, "suggest": 14, "begin": 14, "realiz": 14, "confid": 14, "answer": 14, "pari": 14, "countri": 14, "best": 14, "guess": 14, "franc": 14, "tok": 14, "zip": 14, "topk": 14, "f": [14, 17], "3f": 14, "425": 14, "096": 14, "022": 14, "belgium": 14, "008": 14, "prov": 14, "007": 14, "luxem": 14, "saint": 14, "la": 14, "wai": 14, "corrupt": 14, "represent": 14, "confus": 14, "certain": 14, "clean": 14, "caus": 14, "again": 14, "9": 14, "subject_idx": 14, "fourth": 14, "answer_token": 14, "longer": [14, 17], "002277057385072112": 14, "all_hidden_st": 14, "record": 14, "n_layer": 14, "n_token": 14, "intervened_prob": 14, "_": [14, 15], "rang": 14, "visual": 14, "token_id": 14, "squeez": 14, "token_label": 14, "enumer": 14, "fig": 14, "imshow": 14, "cpu": 14, "y": 14, "color_continuous_midpoint": 14, "color_continuous_scal": 14, "rdbu": 14, "befor": [14, 16], "meng": 14, "et": 14, "al": 14, "2022": 14, "madison": 15, "squar": 15, "garden": 15, "gpt2tokenizerfast": [15, 16, 17], "york": 15, "hidden_states1": 16, "hidden_states2": 16, "hidden_states3": 16, "preliminari": 17, "why": 17, "necessari": 17, "length": 17, "incorrectli": 17, "incorrect_a": 17, "incorrect_b": 17, "shorter": 17, "50256": 17, "464": 17, "reciev": 17, "becaus": 17, "side": 17, "back": 17, "correct_a": 17, "correct_b": 17}, "objects": {"nnsight": [[2, 0, 0, "-", "contexts"], [3, 0, 0, "-", "intervention"], [4, 0, 0, "-", "models"], [5, 0, 0, "-", "module"], [6, 0, 0, "-", "patching"], [7, 0, 0, "-", "tracing"], [8, 0, 0, "-", "util"]], "nnsight.contexts": [[2, 0, 0, "-", "Invoker"], [2, 0, 0, "-", "Runner"], [2, 0, 0, "-", "Tracer"]], "nnsight.contexts.Invoker": [[2, 1, 1, "", "Invoker"]], "nnsight.contexts.Invoker.Invoker": [[2, 2, 1, "", "args"], [2, 2, 1, "", "input"], [2, 2, 1, "", "kwargs"], [2, 3, 1, "", "next"], [2, 3, 1, "", "save_all"], [2, 2, 1, "", "scan"], [2, 2, 1, "", "tracer"]], "nnsight.contexts.Runner": [[2, 1, 1, "", "Runner"]], "nnsight.contexts.Runner.Runner": [[2, 2, 1, "", "blocking"], [2, 2, 1, "", "generation"], [2, 2, 1, "", "remote"], [2, 3, 1, "", "run_local"]], "nnsight.contexts.Tracer": [[2, 1, 1, "", "Tracer"]], "nnsight.contexts.Tracer.Tracer": [[2, 2, 1, "", "args"], [2, 2, 1, "", "batch_size"], [2, 2, 1, "", "batched_input"], [2, 2, 1, "", "generation_idx"], [2, 2, 1, "", "graph"], [2, 2, 1, "", "kwargs"], [2, 2, 1, "", "model"], [2, 2, 1, "", "output"]], "nnsight.intervention": [[3, 1, 1, "", "HookModel"], [3, 1, 1, "", "InterventionProxy"], [3, 1, 1, "", "TokenIndexer"], [3, 5, 1, "", "intervene"]], "nnsight.intervention.HookModel": [[3, 2, 1, "", "handles"], [3, 2, 1, "", "input_hook"], [3, 2, 1, "", "model"], [3, 2, 1, "", "modules"], [3, 2, 1, "", "output_hook"]], "nnsight.intervention.InterventionProxy": [[3, 3, 1, "", "save"], [3, 4, 1, "", "shape"], [3, 4, 1, "", "t"], [3, 4, 1, "", "token"], [3, 4, 1, "", "value"]], "nnsight.models": [[4, 0, 0, "-", "AbstractModel"], [4, 0, 0, "-", "DiffuserModel"], [4, 0, 0, "-", "LanguageModel"]], "nnsight.models.AbstractModel": [[4, 1, 1, "", "AbstractModel"]], "nnsight.models.AbstractModel.AbstractModel": [[4, 2, 1, "", "alter"], [4, 2, 1, "", "args"], [4, 2, 1, "", "custom_model"], [4, 2, 1, "", "dispatch"], [4, 2, 1, "", "dispatched"], [4, 3, 1, "", "forward"], [4, 3, 1, "", "generate"], [4, 3, 1, "", "invoke"], [4, 2, 1, "", "kwargs"], [4, 2, 1, "", "local_model"], [4, 2, 1, "", "meta_model"], [4, 3, 1, "", "modulize"], [4, 2, 1, "", "repoid_path_clsname"]], "nnsight.models.DiffuserModel": [[4, 1, 1, "", "Diffuser"], [4, 1, 1, "", "DiffuserModel"]], "nnsight.models.LanguageModel": [[4, 1, 1, "", "LanguageModel"]], "nnsight.models.LanguageModel.LanguageModel": [[4, 2, 1, "", "config"], [4, 2, 1, "", "local_model"], [4, 2, 1, "", "meta_model"], [4, 2, 1, "", "tokenizer"]], "nnsight.module": [[5, 1, 1, "", "Module"]], "nnsight.module.Module": [[5, 4, 1, "", "backward_input"], [5, 4, 1, "", "backward_output"], [5, 4, 1, "", "input"], [5, 2, 1, "", "input_shape"], [5, 2, 1, "", "input_type"], [5, 2, 1, "", "module_path"], [5, 4, 1, "id1", "output"], [5, 2, 1, "", "output_shape"], [5, 2, 1, "", "output_type"], [5, 2, 1, "", "tracer"], [5, 3, 1, "", "wrap"]], "nnsight.patching": [[6, 1, 1, "", "Patch"], [6, 1, 1, "", "Patcher"]], "nnsight.patching.Patch": [[6, 2, 1, "", "obj"], [6, 2, 1, "", "parent"], [6, 3, 1, "", "patch"], [6, 2, 1, "", "replacement"], [6, 3, 1, "", "restore"]], "nnsight.patching.Patcher": [[6, 3, 1, "", "add"], [6, 2, 1, "", "patches"]], "nnsight.tracing": [[7, 0, 0, "-", "Graph"], [7, 0, 0, "-", "Node"], [7, 0, 0, "-", "Proxy"]], "nnsight.tracing.Graph": [[7, 1, 1, "", "Graph"]], "nnsight.tracing.Graph.Graph": [[7, 3, 1, "", "add"], [7, 2, 1, "", "argument_node_names"], [7, 3, 1, "", "compile"], [7, 2, 1, "", "generation_idx"], [7, 3, 1, "", "increment"], [7, 2, 1, "", "module_proxy"], [7, 2, 1, "", "name_idx"], [7, 2, 1, "", "nodes"], [7, 3, 1, "", "proxy"], [7, 2, 1, "", "proxy_class"], [7, 3, 1, "", "rtn"], [7, 3, 1, "", "trace"], [7, 2, 1, "", "validate"], [7, 3, 1, "", "wrap"]], "nnsight.tracing.Node": [[7, 1, 1, "", "Node"]], "nnsight.tracing.Node.Node": [[7, 2, 1, "", "_proxy_device"], [7, 2, 1, "", "args"], [7, 3, 1, "", "compile"], [7, 2, 1, "", "dependencies"], [7, 3, 1, "", "destroy"], [7, 3, 1, "", "execute"], [7, 3, 1, "", "fulfilled"], [7, 2, 1, "", "graph"], [7, 2, 1, "", "kwargs"], [7, 2, 1, "", "listeners"], [7, 2, 1, "", "meta"], [7, 2, 1, "", "name"], [7, 3, 1, "", "prepare_inputs"], [7, 3, 1, "", "prepare_proxy_values"], [7, 4, 1, "", "proxy_device"], [7, 2, 1, "", "proxy_value"], [7, 3, 1, "", "redundant"], [7, 3, 1, "", "set_value"], [7, 2, 1, "", "target"], [7, 2, 1, "", "value"]], "nnsight.tracing.Proxy": [[7, 1, 1, "", "Proxy"], [7, 5, 1, "", "proxy_wrapper"]], "nnsight.tracing.Proxy.Proxy": [[7, 2, 1, "", "node"], [7, 3, 1, "", "proxy_update"]], "nnsight.util": [[8, 1, 1, "", "WrapperModule"], [8, 5, 1, "", "apply"], [8, 5, 1, "", "cross_entropy_loss"], [8, 5, 1, "", "fetch_attr"], [8, 5, 1, "", "timed"], [8, 5, 1, "", "wrap"]], "nnsight.util.WrapperModule": [[8, 3, 1, "", "forward"]]}, "objtypes": {"0": "py:module", "1": "py:class", "2": "py:attribute", "3": "py:method", "4": "py:property", "5": "py:function"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "class", "Python class"], "2": ["py", "attribute", "Python attribute"], "3": ["py", "method", "Python method"], "4": ["py", "property", "Python property"], "5": ["py", "function", "Python function"]}, "titleterms": {"about": 0, "nnsight": [0, 2, 3, 4, 5, 6, 7, 8, 9], "transpar": 0, "scienc": 0, "api": 0, "black": 0, "box": 0, "infer": 0, "how": 0, "you": 0, "us": 0, "what": 0, "happen": 0, "behind": 0, "scene": 0, "document": 1, "context": 2, "intervent": [3, 15], "model": [4, 14], "modul": 5, "patch": 6, "trace": 7, "util": 8, "get": 10, "start": 10, "instal": 10, "first": 10, "step": 10, "tutori": 11, "basic": 12, "featur": 13, "main": 14, "demo": 14, "load": 14, "run": 14, "exampl": 14, "logit": 14, "len": 14, "interven": 14, "dure": 14, "forward": 14, "pass": 14, "cross": 15, "prompt": 15, "multipl": 16, "token": [16, 17], "gener": 16, "base": 17, "index": 17}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "nbsphinx": 4, "sphinx": 60}, "alltitles": {"About nnsight: a transparent science API for black-box inference": [[0, "about-nnsight-a-transparent-science-api-for-black-box-inference"]], "How you use nnsight": [[0, "how-you-use-nnsight"]], "What happens behind the scenes?": [[0, "what-happens-behind-the-scenes"]], "Documentation": [[1, "documentation"]], "nnsight.contexts": [[2, "module-nnsight.contexts"]], "nnsight.intervention": [[3, "module-nnsight.intervention"]], "nnsight.models": [[4, "module-nnsight.models"]], "nnsight.module": [[5, "module-nnsight.module"]], "nnsight.patching": [[6, "module-nnsight.patching"]], "nnsight.tracing": [[7, "module-nnsight.tracing"]], "nnsight.util": [[8, "module-nnsight.util"]], "nnsight": [[9, "nnsight"]], "Getting Started": [[10, "getting-started"]], "Installation": [[10, "installation"]], "First Steps": [[10, "first-steps"]], "Tutorials": [[11, "tutorials"]], "Basics": [[12, "basics"]], "Features": [[13, "features"]], "Main Demo": [[14, "Main-Demo"]], "Loading and Running Models": [[14, "Loading-and-Running-Models"]], "Example: Logit Lens": [[14, "Example:-Logit-Lens"]], "Intervening during a forward pass": [[14, "Intervening-during-a-forward-pass"]], "Cross-Prompt Intervention": [[15, "Cross-Prompt-Intervention"]], "Multiple Token Generation": [[16, "Multiple-Token-Generation"]], "Token Based Indexing": [[17, "Token-Based-Indexing"]]}, "indexentries": {"invoker (class in nnsight.contexts.invoker)": [[2, "nnsight.contexts.Invoker.Invoker"]], "runner (class in nnsight.contexts.runner)": [[2, "nnsight.contexts.Runner.Runner"]], "tracer (class in nnsight.contexts.tracer)": [[2, "nnsight.contexts.Tracer.Tracer"]], "args (nnsight.contexts.invoker.invoker attribute)": [[2, "nnsight.contexts.Invoker.Invoker.args"]], "args (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.args"]], "batch_size (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.batch_size"]], "batched_input (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.batched_input"]], "blocking (nnsight.contexts.runner.runner attribute)": [[2, "nnsight.contexts.Runner.Runner.blocking"]], "generation (nnsight.contexts.runner.runner attribute)": [[2, "nnsight.contexts.Runner.Runner.generation"]], "generation_idx (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.generation_idx"]], "graph (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.graph"]], "input (nnsight.contexts.invoker.invoker attribute)": [[2, "nnsight.contexts.Invoker.Invoker.input"]], "kwargs (nnsight.contexts.invoker.invoker attribute)": [[2, "nnsight.contexts.Invoker.Invoker.kwargs"]], "kwargs (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.kwargs"]], "model (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.model"]], "module": [[2, "module-nnsight.contexts"], [2, "module-nnsight.contexts.Invoker"], [2, "module-nnsight.contexts.Runner"], [2, "module-nnsight.contexts.Tracer"], [3, "module-nnsight.intervention"], [4, "module-nnsight.models"], [4, "module-nnsight.models.AbstractModel"], [4, "module-nnsight.models.DiffuserModel"], [4, "module-nnsight.models.LanguageModel"], [5, "module-nnsight.module"], [6, "module-nnsight.patching"], [7, "module-nnsight.tracing"], [7, "module-nnsight.tracing.Graph"], [7, "module-nnsight.tracing.Node"], [7, "module-nnsight.tracing.Proxy"], [8, "module-nnsight.util"]], "next() (nnsight.contexts.invoker.invoker method)": [[2, "nnsight.contexts.Invoker.Invoker.next"]], "nnsight.contexts": [[2, "module-nnsight.contexts"]], "nnsight.contexts.invoker": [[2, "module-nnsight.contexts.Invoker"]], "nnsight.contexts.runner": [[2, "module-nnsight.contexts.Runner"]], "nnsight.contexts.tracer": [[2, "module-nnsight.contexts.Tracer"]], "output (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.output"]], "remote (nnsight.contexts.runner.runner attribute)": [[2, "nnsight.contexts.Runner.Runner.remote"]], "run_local() (nnsight.contexts.runner.runner method)": [[2, "nnsight.contexts.Runner.Runner.run_local"]], "save_all() (nnsight.contexts.invoker.invoker method)": [[2, "nnsight.contexts.Invoker.Invoker.save_all"]], "scan (nnsight.contexts.invoker.invoker attribute)": [[2, "nnsight.contexts.Invoker.Invoker.scan"]], "tracer (nnsight.contexts.invoker.invoker attribute)": [[2, "nnsight.contexts.Invoker.Invoker.tracer"]], "hookmodel (class in nnsight.intervention)": [[3, "nnsight.intervention.HookModel"]], "interventionproxy (class in nnsight.intervention)": [[3, "nnsight.intervention.InterventionProxy"]], "tokenindexer (class in nnsight.intervention)": [[3, "nnsight.intervention.TokenIndexer"]], "handles (nnsight.intervention.hookmodel attribute)": [[3, "nnsight.intervention.HookModel.handles"]], "input_hook (nnsight.intervention.hookmodel attribute)": [[3, "nnsight.intervention.HookModel.input_hook"]], "intervene() (in module nnsight.intervention)": [[3, "nnsight.intervention.intervene"]], "model (nnsight.intervention.hookmodel attribute)": [[3, "nnsight.intervention.HookModel.model"]], "modules (nnsight.intervention.hookmodel attribute)": [[3, "nnsight.intervention.HookModel.modules"]], "nnsight.intervention": [[3, "module-nnsight.intervention"]], "output_hook (nnsight.intervention.hookmodel attribute)": [[3, "nnsight.intervention.HookModel.output_hook"]], "save() (nnsight.intervention.interventionproxy method)": [[3, "nnsight.intervention.InterventionProxy.save"]], "shape (nnsight.intervention.interventionproxy property)": [[3, "nnsight.intervention.InterventionProxy.shape"]], "t (nnsight.intervention.interventionproxy property)": [[3, "nnsight.intervention.InterventionProxy.t"]], "token (nnsight.intervention.interventionproxy property)": [[3, "nnsight.intervention.InterventionProxy.token"]], "value (nnsight.intervention.interventionproxy property)": [[3, "nnsight.intervention.InterventionProxy.value"]], "abstractmodel (class in nnsight.models.abstractmodel)": [[4, "nnsight.models.AbstractModel.AbstractModel"]], "diffuser (class in nnsight.models.diffusermodel)": [[4, "nnsight.models.DiffuserModel.Diffuser"]], "diffusermodel (class in nnsight.models.diffusermodel)": [[4, "nnsight.models.DiffuserModel.DiffuserModel"]], "languagemodel (class in nnsight.models.languagemodel)": [[4, "nnsight.models.LanguageModel.LanguageModel"]], "alter (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.alter"]], "args (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.args"]], "config (nnsight.models.languagemodel.languagemodel attribute)": [[4, "nnsight.models.LanguageModel.LanguageModel.config"]], "custom_model (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.custom_model"]], "dispatch (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.dispatch"]], "dispatched (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.dispatched"]], "forward() (nnsight.models.abstractmodel.abstractmodel method)": [[4, "nnsight.models.AbstractModel.AbstractModel.forward"]], "generate() (nnsight.models.abstractmodel.abstractmodel method)": [[4, "nnsight.models.AbstractModel.AbstractModel.generate"]], "invoke() (nnsight.models.abstractmodel.abstractmodel method)": [[4, "nnsight.models.AbstractModel.AbstractModel.invoke"]], "kwargs (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.kwargs"]], "local_model (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.local_model"]], "local_model (nnsight.models.languagemodel.languagemodel attribute)": [[4, "nnsight.models.LanguageModel.LanguageModel.local_model"]], "meta_model (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.meta_model"]], "meta_model (nnsight.models.languagemodel.languagemodel attribute)": [[4, "nnsight.models.LanguageModel.LanguageModel.meta_model"]], "modulize() (nnsight.models.abstractmodel.abstractmodel method)": [[4, "nnsight.models.AbstractModel.AbstractModel.modulize"]], "nnsight.models": [[4, "module-nnsight.models"]], "nnsight.models.abstractmodel": [[4, "module-nnsight.models.AbstractModel"]], "nnsight.models.diffusermodel": [[4, "module-nnsight.models.DiffuserModel"]], "nnsight.models.languagemodel": [[4, "module-nnsight.models.LanguageModel"]], "repoid_path_clsname (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.repoid_path_clsname"]], "tokenizer (nnsight.models.languagemodel.languagemodel attribute)": [[4, "nnsight.models.LanguageModel.LanguageModel.tokenizer"]], "module (class in nnsight.module)": [[5, "nnsight.module.Module"]], "backward_input (nnsight.module.module property)": [[5, "nnsight.module.Module.backward_input"]], "backward_output (nnsight.module.module property)": [[5, "nnsight.module.Module.backward_output"]], "input (nnsight.module.module property)": [[5, "nnsight.module.Module.input"]], "input_shape (nnsight.module.module attribute)": [[5, "nnsight.module.Module.input_shape"]], "input_type (nnsight.module.module attribute)": [[5, "nnsight.module.Module.input_type"]], "module_path (nnsight.module.module attribute)": [[5, "nnsight.module.Module.module_path"]], "nnsight.module": [[5, "module-nnsight.module"]], "output (nnsight.module.module attribute)": [[5, "id0"], [5, "nnsight.module.Module.output"]], "output (nnsight.module.module property)": [[5, "id1"]], "output_shape (nnsight.module.module attribute)": [[5, "nnsight.module.Module.output_shape"]], "output_type (nnsight.module.module attribute)": [[5, "nnsight.module.Module.output_type"]], "tracer (nnsight.module.module attribute)": [[5, "nnsight.module.Module.tracer"]], "wrap() (nnsight.module.module static method)": [[5, "nnsight.module.Module.wrap"]], "patch (class in nnsight.patching)": [[6, "nnsight.patching.Patch"]], "patcher (class in nnsight.patching)": [[6, "nnsight.patching.Patcher"]], "add() (nnsight.patching.patcher method)": [[6, "nnsight.patching.Patcher.add"]], "nnsight.patching": [[6, "module-nnsight.patching"]], "obj (nnsight.patching.patch attribute)": [[6, "nnsight.patching.Patch.obj"]], "parent (nnsight.patching.patch attribute)": [[6, "nnsight.patching.Patch.parent"]], "patch() (nnsight.patching.patch method)": [[6, "nnsight.patching.Patch.patch"]], "patches (nnsight.patching.patcher attribute)": [[6, "nnsight.patching.Patcher.patches"]], "replacement (nnsight.patching.patch attribute)": [[6, "nnsight.patching.Patch.replacement"]], "restore() (nnsight.patching.patch method)": [[6, "nnsight.patching.Patch.restore"]], "graph (class in nnsight.tracing.graph)": [[7, "nnsight.tracing.Graph.Graph"]], "node (class in nnsight.tracing.node)": [[7, "nnsight.tracing.Node.Node"]], "proxy (class in nnsight.tracing.proxy)": [[7, "nnsight.tracing.Proxy.Proxy"]], "_proxy_device (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node._proxy_device"]], "add() (nnsight.tracing.graph.graph method)": [[7, "nnsight.tracing.Graph.Graph.add"]], "args (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.args"]], "argument_node_names (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.argument_node_names"]], "compile() (nnsight.tracing.graph.graph method)": [[7, "nnsight.tracing.Graph.Graph.compile"]], "compile() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.compile"]], "dependencies (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.dependencies"]], "destroy() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.destroy"]], "execute() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.execute"]], "fulfilled() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.fulfilled"]], "generation_idx (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.generation_idx"]], "graph (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.graph"]], "increment() (nnsight.tracing.graph.graph method)": [[7, "nnsight.tracing.Graph.Graph.increment"]], "kwargs (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.kwargs"]], "listeners (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.listeners"]], "meta (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.meta"]], "module_proxy (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.module_proxy"]], "name (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.name"]], "name_idx (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.name_idx"]], "nnsight.tracing": [[7, "module-nnsight.tracing"]], "nnsight.tracing.graph": [[7, "module-nnsight.tracing.Graph"]], "nnsight.tracing.node": [[7, "module-nnsight.tracing.Node"]], "nnsight.tracing.proxy": [[7, "module-nnsight.tracing.Proxy"]], "node (nnsight.tracing.proxy.proxy attribute)": [[7, "nnsight.tracing.Proxy.Proxy.node"]], "nodes (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.nodes"]], "prepare_inputs() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.prepare_inputs"]], "prepare_proxy_values() (nnsight.tracing.node.node static method)": [[7, "nnsight.tracing.Node.Node.prepare_proxy_values"]], "proxy() (nnsight.tracing.graph.graph method)": [[7, "nnsight.tracing.Graph.Graph.proxy"]], "proxy_class (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.proxy_class"]], "proxy_device (nnsight.tracing.node.node property)": [[7, "nnsight.tracing.Node.Node.proxy_device"]], "proxy_update() (nnsight.tracing.proxy.proxy static method)": [[7, "nnsight.tracing.Proxy.Proxy.proxy_update"]], "proxy_value (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.proxy_value"]], "proxy_wrapper() (in module nnsight.tracing.proxy)": [[7, "nnsight.tracing.Proxy.proxy_wrapper"]], "redundant() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.redundant"]], "rtn() (nnsight.tracing.graph.graph static method)": [[7, "nnsight.tracing.Graph.Graph.rtn"]], "set_value() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.set_value"]], "target (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.target"]], "trace() (nnsight.tracing.graph.graph static method)": [[7, "nnsight.tracing.Graph.Graph.trace"]], "validate (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.validate"]], "value (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.value"]], "wrap() (nnsight.tracing.graph.graph method)": [[7, "nnsight.tracing.Graph.Graph.wrap"]], "wrappermodule (class in nnsight.util)": [[8, "nnsight.util.WrapperModule"]], "apply() (in module nnsight.util)": [[8, "nnsight.util.apply"]], "cross_entropy_loss() (in module nnsight.util)": [[8, "nnsight.util.cross_entropy_loss"]], "fetch_attr() (in module nnsight.util)": [[8, "nnsight.util.fetch_attr"]], "forward() (nnsight.util.wrappermodule method)": [[8, "nnsight.util.WrapperModule.forward"]], "nnsight.util": [[8, "module-nnsight.util"]], "timed() (in module nnsight.util)": [[8, "nnsight.util.timed"]], "wrap() (in module nnsight.util)": [[8, "nnsight.util.wrap"]]}})
\ No newline at end of file
+Search.setIndex({"docnames": ["about", "documentation", "documentation/contexts", "documentation/intervention", "documentation/models", "documentation/module", "documentation/patching", "documentation/tracing", "documentation/util", "index", "start", "tutorials", "tutorials/basics", "tutorials/features", "tutorials/main_demo", "tutorials/notebooks/cross_prompt", "tutorials/notebooks/multiple_token", "tutorials/notebooks/token_indexing"], "filenames": ["about.rst", "documentation.rst", "documentation/contexts.rst", "documentation/intervention.rst", "documentation/models.rst", "documentation/module.rst", "documentation/patching.rst", "documentation/tracing.rst", "documentation/util.rst", "index.rst", "start.rst", "tutorials.rst", "tutorials/basics.rst", "tutorials/features.rst", "tutorials/main_demo.ipynb", "tutorials/notebooks/cross_prompt.ipynb", "tutorials/notebooks/multiple_token.ipynb", "tutorials/notebooks/token_indexing.ipynb"], "titles": ["About nnsight", "Documentation", "nnsight.contexts", "nnsight.intervention", "nnsight.models", "nnsight.module", "nnsight.patching", "nnsight.tracing", "nnsight.util", "nnsight", "Getting Started", "Tutorials", "Basics", "Features", "Main Demo", "Cross-Prompt Intervention", "Multiple Token Generation", "Token Based Indexing"], "terms": {"can": [0, 2, 3, 4, 5, 7, 11, 14, 15], "studi": 0, "intern": [0, 9, 10, 14], "deep": [0, 10], "network": [0, 9], "i": [0, 2, 3, 4, 5, 7, 8, 9, 10, 14, 15, 16, 17], "too": 0, "larg": [0, 9], "run": [0, 2, 4, 5, 7, 8, 12], "In": [0, 2, 4, 14, 15, 17], "thi": [0, 2, 3, 4, 5, 7, 8, 10, 14, 15, 17], "era": 0, "scale": 0, "learn": [0, 10, 11], "most": [0, 2, 5], "interest": 0, "model": [0, 1, 2, 3, 5, 7, 9, 10, 12, 15, 16, 17], "ar": [0, 2, 3, 4, 5, 7, 8, 14, 17], "massiv": 0, "hard": 0, "ordinari": 0, "commerci": 0, "infer": [0, 4], "servic": 0, "let": [0, 14, 17], "interact": [0, 3, 4, 7], "huge": 0, "thei": [0, 2, 7], "do": [0, 2, 3, 7], "see": [0, 4, 11, 14, 17], "The": [0, 2, 3, 4, 5, 6, 7, 14, 16, 17], "librari": 0, "differ": [0, 2, 17], "give": [0, 4, 8, 14], "full": 0, "access": [0, 2, 3, 4, 5, 9], "all": [0, 2, 3, 4, 5, 7, 8], "neural": [0, 9], "when": [0, 2, 3, 4, 7, 16, 17], "togeth": [0, 2], "remot": [0, 2], "like": [0, 2, 4, 9, 10, 14], "nation": 0, "facil": 0, "ndif": [0, 2], "expriment": 0, "open": 0, "easili": [0, 3], "also": [0, 2, 4, 5, 6, 11, 14, 15], "terrif": 0, "smaller": 0, "local": [0, 4], "overview": 0, "pipelin": 0, "research": 0, "write": 0, "simpl": [0, 4, 8], "python": [0, 7, 17], "code": [0, 9], "along": [0, 3], "unlik": 0, "experi": [0, 10], "read": 0, "ani": [0, 2, 3, 4, 6, 7, 8, 9, 14], "state": [0, 3, 4, 14, 16, 17], "being": [0, 7, 14], "creat": [0, 2, 4, 5, 7, 14], "comput": [0, 2, 3, 4, 7, 8, 14], "graph": [0, 2, 3, 4, 5, 7], "sent": 0, "interleav": [0, 3, 4, 7], "execut": [0, 2, 3, 4, 5, 7, 8, 14], "built": [0, 2, 7], "pytorch": [0, 4], "veri": 0, "similar": 0, "your": [0, 14], "own": [0, 7], "workstataion": 0, "fact": 0, "same": [0, 2, 7, 8, 15], "small": 0, "just": [0, 7, 17], "chang": [0, 14], "few": [0, 14], "argument": [0, 2, 3, 4, 7, 8], "between": [0, 15], "normal": [0, 4, 17], "treat": [0, 14], "opaqu": 0, "instead": [0, 8], "set": [0, 2, 3, 5, 7, 9], "up": [0, 9], "context": [0, 1, 3, 4, 5, 6, 9, 14], "enabl": [0, 3, 4, 9], "get": [0, 2, 5, 7, 8, 9, 14, 15, 16, 17], "direct": [0, 4, 7, 14], "while": [0, 8, 16], "here": [0, 2, 4, 14, 15, 16], "look": 0, "from": [0, 2, 3, 4, 6, 7, 8, 9, 14, 15, 16, 17], "import": [0, 4, 6, 11, 14, 15, 16, 17], "languagemodel": [0, 2, 4, 14, 15, 16, 17], "meta": [0, 2, 4, 5, 7], "llama": 0, "2": [0, 14, 15, 16, 17], "70b": 0, "hf": 0, "forward": [0, 2, 4, 5, 7, 8, 12, 17], "true": [0, 2, 4, 7, 8, 14], "runner": [0, 2, 3, 4, 14, 17], "invok": [0, 2, 3, 4, 5, 14, 15, 16, 17], "eiffel": [0, 2, 3, 4, 5, 16, 17], "tower": [0, 2, 3, 4, 5, 16, 17], "citi": [0, 2, 3, 4, 5, 14, 15, 16, 17], "hidden_st": [0, 2, 3, 5, 17], "layer": [0, 14, 16], "10": [0, 14, 16], "input": [0, 2, 3, 4, 5, 8, 14, 17], "save": [0, 2, 3, 4, 5, 14, 15, 16, 17], "one": [0, 4, 5, 7, 8, 14, 16], "hidden": [0, 3, 14, 16, 17], "11": [0, 4, 14], "mlp": [0, 3, 4, 5, 14], "output": [0, 2, 3, 4, 5, 7, 14, 15, 16, 17], "0": [0, 2, 3, 4, 5, 7, 14, 15, 16, 17], "modul": [0, 1, 2, 3, 4, 6, 7, 8, 9, 14], "print": [0, 2, 3, 4, 5, 8, 14, 15, 16, 17], "predict": [0, 4], "wa": [0, 4, 14], "valu": [0, 2, 3, 4, 5, 7, 14, 15, 16, 17], "easi": [0, 2], "huggingfac": [0, 4, 9, 14], "load": [0, 4, 12], "object": [0, 2, 3, 4, 5, 6, 7, 8], "wee": 0, "line": [0, 9, 14], "notic": [0, 17], "we": [0, 4, 5, 7, 14, 15, 16, 17], "70": 0, "billion": 0, "paramet": [0, 2, 3, 4, 5, 6, 7, 8, 14], "which": [0, 2, 4, 5, 7, 8, 14], "ordinarili": 0, "pretti": 0, "difficult": 0, "regular": 0, "workstat": 0, "sinc": [0, 8, 14], "would": [0, 3, 14], "take": [0, 3, 8], "140": 0, "280": 0, "gigabyt": 0, "gpu": [0, 4, 14], "ram": 0, "store": [0, 3, 7, 14], "trick": 0, "u": 0, "work": [0, 2, 3, 14, 15], "3": [0, 3, 4, 14, 15, 16, 17], "flag": 0, "indic": [0, 3, 14], "want": [0, 2, 14], "actual": [0, 2, 3, 4, 7, 14], "By": 0, "default": [0, 2, 4, 7, 8], "If": [0, 2, 3, 4, 7, 8, 10, 14], "quickli": 0, "could": [0, 3, 15], "leav": 0, "fals": [0, 2, 4, 8, 14], "Then": 0, "4": [0, 14, 17], "call": [0, 2, 3, 4, 5, 6, 7, 8, 14, 15, 16, 17], "function": [0, 2, 3, 4, 5, 6, 7, 8, 11, 17], "manag": [0, 2, 3, 4, 6], "reason": 0, "doe": [0, 5, 14], "provid": [0, 2, 3, 4], "5": [0, 14], "6": [0, 14], "On": [0, 2], "grab": [0, 15], "insid": [0, 14], "transform": [0, 2, 3, 4, 5, 8, 9, 15, 16, 17], "block": [0, 2, 9, 15], "calcul": 0, "outermost": 0, "complet": [0, 2, 3, 7, 14], "defin": [0, 8], "cacul": 0, "server": [0, 2], "": [0, 2, 3, 4, 5, 7, 8, 14, 17], "done": [0, 3], "result": [0, 2, 3, 5, 7, 14, 17], "shown": [0, 15], "7": [0, 14], "8": [0, 14], "help": 0, "understand": 0, "oper": [0, 2, 4, 5, 7, 14, 15], "immedi": 0, "add": [0, 2, 3, 4, 5, 6, 7], "intervent": [0, 1, 2, 4, 5, 9, 13, 16], "alongsid": 0, "upon": [0, 3], "exit": [0, 2, 3, 14], "exampl": [0, 2, 3, 4, 5, 10, 12, 17], "seen": 0, "below": [0, 2, 5, 11], "node": [0, 3, 5, 7], "depend": [0, 7], "well": [0, 2, 7], "other": [0, 4, 7], "chosen": [0, 2], "request": [0, 5], "inject": 0, "basic": [0, 10, 11], "lot": 0, "insight": 0, "go": [0, 4], "For": [0, 3, 14], "logit": [0, 2, 3, 4, 5, 8, 12], "len": [0, 12], "text": [0, 14, 15, 16, 17], "And": [0, 6], "causal": [0, 14], "trace": [0, 1, 2, 3, 4], "path": [0, 3, 4, 5, 8], "patch": [0, 1], "circuit": 0, "discoveri": 0, "method": [0, 2, 3, 4, 7, 8, 14, 15, 16, 17], "locat": [0, 14, 15], "compon": [0, 14], "within": [0, 2, 4, 5, 8, 14, 15], "plai": 0, "decis": 0, "role": 0, "make": [0, 2, 3, 14], "thes": 0, "provi": 0, "gradient": [0, 9], "optim": 0, "out": [0, 2, 4, 6, 7, 10, 11, 14], "order": [0, 2, 3, 4, 5, 7], "applic": 0, "cross": [0, 8, 13], "prompt": [0, 2, 4, 13, 14, 16], "mani": [0, 2, 7], "more": [0, 4, 7, 10, 11, 14, 16], "featur": [0, 2, 8, 9, 10, 11], "page": 0, "inform": [0, 2, 4, 7, 14], "project": 0, "current": [0, 2, 3, 7], "alpha": 0, "pre": [0, 2, 7, 15], "releas": 0, "earli": 0, "user": [0, 3, 4, 5], "contributor": 0, "contribut": 0, "join": 0, "discord": 0, "updat": [0, 2, 3, 7, 8], "bug": 0, "report": 0, "opportun": 0, "effort": 0, "nnsight": [1, 10, 11, 14, 15, 16, 17], "util": 1, "contain": [2, 3, 4], "logic": [2, 3], "primari": [2, 4], "two": [2, 4, 15, 17], "class": [2, 3, 4, 5, 6, 7, 8], "involv": [2, 3, 7], "tracer": [2, 5], "around": [2, 3, 4], "meta_model": [2, 4], "an": [2, 3, 4, 5, 6, 7, 8, 14], "abstractmodel": [2, 4, 5], "track": [2, 4, 7, 14], "perform": [2, 4, 5, 7, 8, 14, 17], "said": [2, 7], "expos": 2, "attribut": [2, 3, 4, 5, 6, 7, 8], "To": [2, 10, 14], "need": [2, 4, 7, 8], "know": 2, "about": [2, 14], "so": [2, 3, 7, 14, 17], "each": [2, 7, 14, 16, 17], "keep": 2, "batch_siz": [2, 3], "recent": [2, 5], "gener": [2, 3, 4, 5, 7, 13, 15], "index": [2, 3, 7, 13, 14], "multi": [2, 4], "iter": [2, 3, 4], "made": 2, "dure": [2, 3, 5, 7, 12], "its": [2, 4, 7, 9], "batched_input": 2, "ad": [2, 3, 4, 7], "should": [2, 3, 7, 8, 16], "format": 2, "where": [2, 4, 7], "batch": [2, 3, 4, 8], "allow": [2, 3, 4, 7], "thing": 2, "consist": [2, 14], "valid": [2, 7], "both": [2, 3, 4, 17], "becom": [2, 14], "case": [2, 4, 14, 15], "regardless": 2, "whether": 2, "string": [2, 5, 7, 8, 17], "process": [2, 7], "dictionari": 2, "id": [2, 4, 8, 9, 14], "onli": [2, 3, 4, 5, 7, 8, 9, 14], "us": [2, 3, 4, 7, 8, 14, 15, 16, 17], "carri": [2, 6, 7], "what": [2, 7], "accept": [2, 4], "parent": [2, 6, 7], "appropri": 2, "respect": [2, 5], "enter": [2, 4, 15], "some": [2, 7, 8, 10, 11, 14], "leverag": [2, 4], "prepar": [2, 4, 7], "version": [2, 4, 7], "size": [2, 3, 5, 8, 14, 16], "It": [2, 4, 7], "through": [2, 3, 4, 5, 7, 8, 11, 14], "dtype": [2, 5], "base": [2, 3, 7, 13], "characterist": 2, "come": [2, 11, 15], "implement": [2, 4, 7], "_gener": [2, 4], "_forward": [2, 4], "rich": [2, 17], "new": [2, 7, 15], "eventu": 2, "ha": [2, 4, 7, 8], "assum": 2, "arg": [2, 3, 4, 7, 8], "bool": [2, 4, 7, 8], "kwarg": [2, 4, 7, 8], "given": [2, 7, 8, 14], "_run_loc": 2, "show": [2, 5, 14], "gpt2": [2, 4, 5, 15, 16, 17], "them": [2, 3, 4, 5, 7, 8, 15], "after": [2, 4, 8, 14, 16], "lm_head": [2, 3, 4, 5], "zero": [2, 5, 14], "h": [2, 3, 4, 5, 8, 16, 17], "otherwis": [2, 5, 7], "type": [2, 3, 4, 5, 6, 7, 8], "option": [2, 4, 7, 8], "hang": 2, "until": [2, 14], "job": 2, "return": [2, 3, 4, 5, 7, 8, 14, 16], "you": [2, 3, 4, 7, 10, 11, 14, 15, 16, 17], "retriev": [2, 3, 8], "run_loc": 2, "local_model": [2, 4], "th": 2, "later": [2, 14], "posit": [2, 3, 4, 7, 14], "pass": [2, 4, 5, 7, 8, 9, 12], "list": [2, 3, 4, 5, 6, 7, 14], "keyword": [2, 4, 7], "dict": [2, 4, 7, 14], "str": [2, 3, 4, 5, 7, 8], "proxi": [2, 3, 5, 7, 14], "int": [2, 4, 7], "generation_idx": [2, 7], "idx": [2, 3, 17], "__exit__": [2, 3, 6], "scan": 2, "meant": [2, 4], "tandem": [2, 4], "initi": [2, 4, 5, 16], "post": [2, 7], "_prepare_input": 2, "_scan": 2, "potenti": [2, 3], "correctli": [2, 14, 17], "free": [2, 7], "wise": 2, "mai": 2, "turn": 2, "loop": 2, "shape": [2, 3, 5, 7, 8, 14, 16], "error": [2, 7], "have": [2, 3, 7, 15, 17], "first": [2, 3, 4, 5, 7, 8, 14, 15, 17], "next": [2, 16], "increment": [2, 7], "1": [2, 4, 8, 14, 15, 16, 17], "none": [2, 3, 4, 6, 7], "design": [2, 3], "subsequ": [2, 3, 16], "appli": [2, 3, 4, 8, 10, 11, 14, 16], "how": [2, 5, 10, 11, 14, 16, 17], "onc": 2, "save_al": 2, "module_path": [2, 3, 5], "kei": [2, 3, 4, 7, 9, 11], "interventionproxi": [3, 5], "extend": 3, "fx": 3, "easier": 3, "interven": [3, 12], "entri": [3, 7], "hook": [3, 4, 5, 7, 8], "hookmodel": 3, "remov": [3, 7], "module_kei": 3, "input_hook": 3, "callabl": [3, 7, 8], "output_hook": 3, "backward_input_hook": 3, "backward_output_hook": 3, "regist": [3, 8], "__enter__": [3, 6], "root": [3, 4, 5, 7, 8], "torch": [3, 4, 5, 7, 8, 14, 16], "nn": [3, 4, 5, 7], "tupl": [3, 7, 14], "signatur": 3, "handl": [3, 6], "removablehandl": 3, "sub": [3, 4], "addit": 3, "delet": 3, "listen": [3, 7], "had": 3, "been": [3, 8], "token": [3, 4, 8, 13, 14, 15], "t": [3, 14, 17], "multipl": [3, 7, 13, 17], "pad": [3, 14, 15, 16, 17], "mean": [3, 14], "shift": [3, 8], "care": [3, 8], "collect": [3, 7, 8], "tensor": [3, 4, 5, 7, 8, 15, 17], "popul": [3, 5, 7], "properti": [3, 5, 7], "union": [3, 4, 7], "tokenindex": 3, "alia": 3, "directli": [3, 4], "second": [3, 15], "dimens": 3, "neg": 3, "left": [3, 17], "equival": 3, "helper": [3, 8, 17], "aid": 3, "activ": [3, 4, 9, 14], "form": [3, 4, 7], "check": [3, 4, 7, 10, 11], "argument_node_nam": [3, 7], "exist": [3, 5], "name": [3, 4, 7], "batch_start": 3, "narrow": 3, "select": 3, "relev": [3, 14], "view": [3, 7, 14], "oppos": 3, "copi": 3, "downstream": [3, 4, 5], "edit": 3, "origin": [3, 6, 14], "prevent": 3, "effect": 3, "bath": 3, "outsid": [3, 7], "preview": 3, "either": [3, 7], "rel": [3, 5], "denot": [3, 5, 16], "modifi": 3, "main": [4, 11, 12], "wrap": [4, 5, 7, 8], "repo": [4, 9, 14], "device_map": [4, 14, 15, 16, 17], "cuda": [4, 14, 15, 16, 17], "declar": 4, "entail": 4, "underli": [4, 14], "automodelforcausallm": 4, "unus": 4, "acceler": 4, "packag": [4, 8, 9, 10, 11], "encompass": 4, "displai": 4, "structur": [4, 14], "abil": 4, "itself": 4, "gpt2lmheadmodel": 4, "gpt2model": 4, "wte": [4, 15], "embed": [4, 14, 15], "50257": 4, "768": [4, 16], "wpe": 4, "1024": 4, "drop": 4, "dropout": [4, 14], "p": [4, 14], "inplac": [4, 14], "modulelist": [4, 5, 14], "12": [4, 14], "x": [4, 5, 8, 14], "gpt2block": 4, "ln_1": 4, "layernorm": [4, 14], "ep": [4, 14], "1e": [4, 14], "05": [4, 14], "elementwise_affin": [4, 14], "attn": 4, "gpt2attentionalt": 4, "c_attn": 4, "conv1d": 4, "c_proj": 4, "attn_dropout": 4, "resid_dropout": 4, "queri": 4, "wrappermodul": [4, 8], "ln_2": 4, "gpt2mlp": 4, "c_fc": 4, "act": [4, 5, 7, 14], "newgeluactiv": 4, "ln_f": 4, "linear": [4, 14], "in_featur": [4, 14], "out_featur": [4, 14], "bia": [4, 14], "determin": 4, "behavior": 4, "three": [4, 16], "invoc": [4, 15, 17], "languag": [4, 14], "over": [4, 14], "max_new_token": [4, 15, 16], "logits1": 4, "empir": 4, "build": 4, "logits2": 4, "repoid_path_model": 4, "dispatch": 4, "alter": 4, "abstract": 4, "wish": [4, 5], "gain": 4, "repoid_path_clsnam": 4, "hug": 4, "face": 4, "checkpoint": 4, "custom": 4, "bee": 4, "yet": [4, 14], "init": 4, "custom_model": 4, "devic": [4, 7, 17], "ran": [4, 7], "specif": [4, 5, 14, 17], "ultim": 4, "A": [4, 7], "doc": [4, 9], "limit": 4, "amount": 4, "anf": 4, "extra": 4, "fwd_arg": 4, "rtype": 4, "node_nam": 4, "module_nam": 4, "_summary_": 4, "_description_": [4, 7], "wrapper": [4, 8], "config": 4, "file": 4, "repositori": [4, 10], "pretrainedconfig": 4, "lm": 4, "pretrainedtoken": 4, "pretrainedmodel": 4, "diffusermodel": 4, "diffus": 4, "entrypoint": 5, "interwoven": 5, "manipul": [5, 9, 10], "requir": 5, "correct": [5, 7, 14, 17], "tree": [5, 14], "repres": [5, 6, 7], "separ": [5, 8], "e": [5, 7, 8], "reset": [5, 7], "output_shap": 5, "nest": 5, "output_typ": 5, "input_shap": 5, "input_typ": 5, "must": [5, 7], "manual": 5, "backward_input": 5, "therefor": [5, 7], "time": [5, 8, 14], "refer": [5, 7], "alreadi": [5, 14], "backward_output": 5, "static": [5, 7], "our": [5, 14], "data": [5, 7, 8, 14], "obj": 6, "replac": [6, 7, 15], "importlib": 6, "import_modul": 6, "self": 6, "__module__": 6, "setattr": 6, "__name__": 6, "restor": [6, 14], "patcher": 6, "point": [7, 9], "flow": 7, "AND": 7, "proxy_valu": 7, "datatyp": 7, "set_valu": 7, "bi": 7, "convert": 7, "respons": 7, "similarli": 7, "ceas": 7, "destroi": 7, "memori": 7, "re": [7, 14, 15, 16, 17], "remain": 7, "traceabl": 7, "magic": 7, "__torch_function__": 7, "find": 7, "shame": 7, "proxy_class": 7, "reserv": 7, "target": [7, 8], "There": 7, "singl": 7, "__init__": [7, 8], "compil": 7, "whatev": 7, "Their": 7, "map": 7, "These": [7, 14], "kick": 7, "rtn": 7, "null": 7, "never": 7, "possibl": 7, "etc": 7, "name_idx": 7, "target_nam": 7, "number": 7, "previou": 7, "uniqu": 7, "module_proxi": 7, "pull": 7, "rais": 7, "valueerror": 7, "than": [7, 14, 15, 16, 17], "final": [7, 14], "module_0": 7, "consid": [7, 14], "specifi": [7, 14], "_type_": 7, "desc": 7, "action": 7, "whole": 7, "_proxy_devic": 7, "remaining_listen": 7, "remaining_depend": 7, "log": 7, "destruct": 7, "noth": 7, "fulfil": 7, "prepare_input": 7, "move": [7, 14], "prepare_proxy_valu": 7, "proxy_devic": 7, "lazi": 7, "creation": 7, "redund": 7, "event": 7, "now": [7, 14, 17], "proxy_upd": 7, "value1": 7, "value2": 7, "proxy_wrapp": 7, "fn": [7, 8], "problemat": 7, "sometim": 7, "throughout": 8, "element": 8, "everi": 8, "overridden": 8, "subclass": 8, "although": 8, "recip": 8, "instanc": 8, "afterward": 8, "former": 8, "latter": 8, "silent": 8, "ignor": 8, "cl": 8, "member": 8, "kind": 8, "cross_entropy_loss": 8, "target_id": 8, "avg_batch": 8, "avg_token": 8, "entropi": 8, "loss": 8, "n": 8, "last": [8, 16], "averag": 8, "across": [8, 16], "fetch_attr": 8, "hierarchi": 8, "level": 8, "fetch": 8, "func": 8, "lggr": 8, "decor": 8, "__class__": 8, "interpret": [9, 10, 11], "\u025bn": [9, 10], "sa\u026at": [9, 10], "start": [9, 14], "tutori": [9, 10], "integr": 9, "weight": [9, 14], "intuit": 9, "coupl": 9, "interoper": 9, "grad": 9, "train": 9, "lora": 9, "probe": 9, "pip": 10, "section": 10, "colab": 10, "d": 10, "hand": [10, 14], "cover": 10, "demo": [11, 12], "quick": 11, "walkthrough": 11, "walk": 11, "advanc": 11, "toolkit": 11, "familiar": 11, "yourself": 11, "tool": 11, "soon": 11, "paper": 11, "pretrain": 14, "ll": 14, "eleutherai": 14, "1b": 14, "pythia": 14, "onto": 14, "abov": 14, "command": 14, "download": 14, "configur": 14, "machin": 14, "even": 14, "though": 14, "still": 14, "inspect": 14, "gptneoxforcausallm": 14, "gpt_neox": 14, "gptneoxmodel": 14, "embed_in": 14, "50304": 14, "2048": 14, "emb_dropout": 14, "15": 14, "16": 14, "gptneoxlay": 14, "input_layernorm": 14, "post_attention_layernorm": 14, "post_attention_dropout": 14, "post_mlp_dropout": 14, "attent": 14, "gptneoxattent": 14, "rotary_emb": 14, "gptneoxrotaryembed": 14, "query_key_valu": 14, "6144": 14, "dens": 14, "attention_dropout": 14, "gptneoxmlp": 14, "dense_h_to_4h": 14, "8192": 14, "dense_4h_to_h": 14, "geluactiv": 14, "final_layer_norm": 14, "embed_out": 14, "extract": 14, "residu": 14, "stream": 14, "john": 14, "mari": 14, "went": 14, "milk": 14, "last_hidden_st": 14, "aren": 14, "39": [14, 15, 16, 17], "gptneoxtokenizerfast": 14, "pleas": [14, 15, 16, 17], "note": [14, 15, 16, 17], "fast": [14, 15, 16, 17], "__call__": [14, 15, 16, 17], "faster": [14, 15, 16, 17], "encod": [14, 15, 16, 17], "follow": [14, 15, 16, 17], "13": 14, "break": 14, "down": 14, "tell": 14, "modif": 14, "detail": 14, "cach": 14, "desir": 14, "As": [14, 17], "probabl": 14, "One": 14, "techniqu": 14, "decod": [14, 15], "intermedi": 14, "crude": 14, "approxim": 14, "def": 14, "norm": 14, "unembed": 14, "all_prob": 14, "prob": 14, "softmax": 14, "dim": 14, "append": 14, "distribut": 14, "evolv": 14, "plotli": 14, "express": 14, "px": 14, "mary_token": 14, "input_id": 14, "layer_prob": 14, "item": 14, "titl": 14, "accord": 14, "label": 14, "suggest": 14, "begin": 14, "realiz": 14, "confid": 14, "answer": 14, "pari": 14, "countri": 14, "best": 14, "guess": 14, "franc": 14, "tok": 14, "zip": 14, "topk": 14, "f": [14, 17], "3f": 14, "425": 14, "096": 14, "022": 14, "belgium": 14, "008": 14, "prov": 14, "007": 14, "luxem": 14, "saint": 14, "la": 14, "wai": 14, "corrupt": 14, "represent": 14, "confus": 14, "certain": 14, "clean": 14, "caus": 14, "again": 14, "9": 14, "subject_idx": 14, "fourth": 14, "answer_token": 14, "longer": [14, 17], "002277057385072112": 14, "all_hidden_st": 14, "record": 14, "n_layer": 14, "n_token": 14, "intervened_prob": 14, "_": [14, 15], "rang": 14, "visual": 14, "token_id": 14, "squeez": 14, "token_label": 14, "enumer": 14, "fig": 14, "imshow": 14, "cpu": 14, "y": 14, "color_continuous_midpoint": 14, "color_continuous_scal": 14, "rdbu": 14, "befor": [14, 16], "meng": 14, "et": 14, "al": 14, "2022": 14, "madison": 15, "squar": 15, "garden": 15, "gpt2tokenizerfast": [15, 16, 17], "york": 15, "hidden_states1": 16, "hidden_states2": 16, "hidden_states3": 16, "preliminari": 17, "why": 17, "necessari": 17, "length": 17, "incorrectli": 17, "incorrect_a": 17, "incorrect_b": 17, "shorter": 17, "50256": 17, "464": 17, "reciev": 17, "becaus": 17, "side": 17, "back": 17, "correct_a": 17, "correct_b": 17}, "objects": {"nnsight": [[2, 0, 0, "-", "contexts"], [3, 0, 0, "-", "intervention"], [4, 0, 0, "-", "models"], [5, 0, 0, "-", "module"], [6, 0, 0, "-", "patching"], [7, 0, 0, "-", "tracing"], [8, 0, 0, "-", "util"]], "nnsight.contexts": [[2, 0, 0, "-", "Invoker"], [2, 0, 0, "-", "Runner"], [2, 0, 0, "-", "Tracer"]], "nnsight.contexts.Invoker": [[2, 1, 1, "", "Invoker"]], "nnsight.contexts.Invoker.Invoker": [[2, 2, 1, "", "args"], [2, 2, 1, "", "input"], [2, 2, 1, "", "kwargs"], [2, 3, 1, "", "next"], [2, 3, 1, "", "save_all"], [2, 2, 1, "", "scan"], [2, 2, 1, "", "tracer"]], "nnsight.contexts.Runner": [[2, 1, 1, "", "Runner"]], "nnsight.contexts.Runner.Runner": [[2, 2, 1, "", "blocking"], [2, 2, 1, "", "generation"], [2, 2, 1, "", "remote"], [2, 3, 1, "", "run_local"]], "nnsight.contexts.Tracer": [[2, 1, 1, "", "Tracer"]], "nnsight.contexts.Tracer.Tracer": [[2, 2, 1, "", "args"], [2, 2, 1, "", "batch_size"], [2, 2, 1, "", "batched_input"], [2, 2, 1, "", "generation_idx"], [2, 2, 1, "", "graph"], [2, 2, 1, "", "kwargs"], [2, 2, 1, "", "model"], [2, 2, 1, "", "output"]], "nnsight.intervention": [[3, 1, 1, "", "HookModel"], [3, 1, 1, "", "InterventionProxy"], [3, 1, 1, "", "TokenIndexer"], [3, 5, 1, "", "intervene"]], "nnsight.intervention.HookModel": [[3, 2, 1, "", "handles"], [3, 2, 1, "", "input_hook"], [3, 2, 1, "", "model"], [3, 2, 1, "", "modules"], [3, 2, 1, "", "output_hook"]], "nnsight.intervention.InterventionProxy": [[3, 3, 1, "", "save"], [3, 4, 1, "", "shape"], [3, 4, 1, "", "t"], [3, 4, 1, "", "token"], [3, 4, 1, "", "value"]], "nnsight.models": [[4, 0, 0, "-", "AbstractModel"], [4, 0, 0, "-", "DiffuserModel"], [4, 0, 0, "-", "LanguageModel"]], "nnsight.models.AbstractModel": [[4, 1, 1, "", "AbstractModel"]], "nnsight.models.AbstractModel.AbstractModel": [[4, 2, 1, "", "alter"], [4, 2, 1, "", "args"], [4, 2, 1, "", "custom_model"], [4, 2, 1, "", "dispatch"], [4, 2, 1, "", "dispatched"], [4, 3, 1, "", "forward"], [4, 3, 1, "", "generate"], [4, 3, 1, "", "invoke"], [4, 2, 1, "", "kwargs"], [4, 2, 1, "", "local_model"], [4, 2, 1, "", "meta_model"], [4, 3, 1, "", "modulize"], [4, 2, 1, "", "repoid_path_clsname"]], "nnsight.models.DiffuserModel": [[4, 1, 1, "", "Diffuser"], [4, 1, 1, "", "DiffuserModel"]], "nnsight.models.LanguageModel": [[4, 1, 1, "", "LanguageModel"]], "nnsight.models.LanguageModel.LanguageModel": [[4, 2, 1, "", "config"], [4, 2, 1, "", "local_model"], [4, 2, 1, "", "meta_model"], [4, 2, 1, "", "tokenizer"]], "nnsight.module": [[5, 1, 1, "", "Module"]], "nnsight.module.Module": [[5, 4, 1, "", "backward_input"], [5, 4, 1, "", "backward_output"], [5, 4, 1, "", "input"], [5, 2, 1, "", "input_shape"], [5, 2, 1, "", "input_type"], [5, 2, 1, "", "module_path"], [5, 4, 1, "id1", "output"], [5, 2, 1, "", "output_shape"], [5, 2, 1, "", "output_type"], [5, 2, 1, "", "tracer"], [5, 3, 1, "", "wrap"]], "nnsight.patching": [[6, 1, 1, "", "Patch"], [6, 1, 1, "", "Patcher"]], "nnsight.patching.Patch": [[6, 2, 1, "", "obj"], [6, 2, 1, "", "parent"], [6, 3, 1, "", "patch"], [6, 2, 1, "", "replacement"], [6, 3, 1, "", "restore"]], "nnsight.patching.Patcher": [[6, 3, 1, "", "add"], [6, 2, 1, "", "patches"]], "nnsight.tracing": [[7, 0, 0, "-", "Graph"], [7, 0, 0, "-", "Node"], [7, 0, 0, "-", "Proxy"]], "nnsight.tracing.Graph": [[7, 1, 1, "", "Graph"]], "nnsight.tracing.Graph.Graph": [[7, 3, 1, "", "add"], [7, 2, 1, "", "argument_node_names"], [7, 3, 1, "", "compile"], [7, 2, 1, "", "generation_idx"], [7, 3, 1, "", "increment"], [7, 2, 1, "", "module_proxy"], [7, 2, 1, "", "name_idx"], [7, 2, 1, "", "nodes"], [7, 3, 1, "", "proxy"], [7, 2, 1, "", "proxy_class"], [7, 3, 1, "", "rtn"], [7, 3, 1, "", "trace"], [7, 2, 1, "", "validate"], [7, 3, 1, "", "wrap"]], "nnsight.tracing.Node": [[7, 1, 1, "", "Node"]], "nnsight.tracing.Node.Node": [[7, 2, 1, "", "_proxy_device"], [7, 2, 1, "", "args"], [7, 3, 1, "", "compile"], [7, 2, 1, "", "dependencies"], [7, 3, 1, "", "destroy"], [7, 3, 1, "", "execute"], [7, 3, 1, "", "fulfilled"], [7, 2, 1, "", "graph"], [7, 2, 1, "", "kwargs"], [7, 2, 1, "", "listeners"], [7, 2, 1, "", "meta"], [7, 2, 1, "", "name"], [7, 3, 1, "", "prepare_inputs"], [7, 3, 1, "", "prepare_proxy_values"], [7, 4, 1, "", "proxy_device"], [7, 2, 1, "", "proxy_value"], [7, 3, 1, "", "redundant"], [7, 3, 1, "", "set_value"], [7, 2, 1, "", "target"], [7, 2, 1, "", "value"]], "nnsight.tracing.Proxy": [[7, 1, 1, "", "Proxy"], [7, 5, 1, "", "proxy_wrapper"]], "nnsight.tracing.Proxy.Proxy": [[7, 2, 1, "", "node"], [7, 3, 1, "", "proxy_update"]], "nnsight.util": [[8, 1, 1, "", "WrapperModule"], [8, 5, 1, "", "apply"], [8, 5, 1, "", "cross_entropy_loss"], [8, 5, 1, "", "fetch_attr"], [8, 5, 1, "", "timed"], [8, 5, 1, "", "wrap"]], "nnsight.util.WrapperModule": [[8, 3, 1, "", "forward"]]}, "objtypes": {"0": "py:module", "1": "py:class", "2": "py:attribute", "3": "py:method", "4": "py:property", "5": "py:function"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "class", "Python class"], "2": ["py", "attribute", "Python attribute"], "3": ["py", "method", "Python method"], "4": ["py", "property", "Python property"], "5": ["py", "function", "Python function"]}, "titleterms": {"about": 0, "nnsight": [0, 2, 3, 4, 5, 6, 7, 8, 9], "an": 0, "api": 0, "transpar": 0, "scienc": 0, "black": 0, "box": 0, "ai": 0, "how": 0, "you": 0, "us": 0, "what": 0, "happen": 0, "behind": 0, "scene": 0, "document": 1, "context": 2, "intervent": [3, 15], "model": [4, 14], "modul": 5, "patch": 6, "trace": 7, "util": 8, "get": 10, "start": 10, "instal": 10, "first": 10, "step": 10, "tutori": 11, "basic": 12, "featur": 13, "main": 14, "demo": 14, "load": 14, "run": 14, "exampl": 14, "logit": 14, "len": 14, "interven": 14, "dure": 14, "forward": 14, "pass": 14, "cross": 15, "prompt": 15, "multipl": 16, "token": [16, 17], "gener": 16, "base": 17, "index": 17}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "nbsphinx": 4, "sphinx": 60}, "alltitles": {"About nnsight": [[0, "about-nnsight"]], "An API for transparent science on black-box AI": [[0, "an-api-for-transparent-science-on-black-box-ai"]], "How you use nnsight": [[0, "how-you-use-nnsight"]], "What happens behind the scenes?": [[0, "what-happens-behind-the-scenes"]], "Documentation": [[1, "documentation"]], "nnsight.contexts": [[2, "module-nnsight.contexts"]], "nnsight.intervention": [[3, "module-nnsight.intervention"]], "nnsight.models": [[4, "module-nnsight.models"]], "nnsight.module": [[5, "module-nnsight.module"]], "nnsight.patching": [[6, "module-nnsight.patching"]], "nnsight.tracing": [[7, "module-nnsight.tracing"]], "nnsight.util": [[8, "module-nnsight.util"]], "nnsight": [[9, "nnsight"]], "Getting Started": [[10, "getting-started"]], "Installation": [[10, "installation"]], "First Steps": [[10, "first-steps"]], "Tutorials": [[11, "tutorials"]], "Basics": [[12, "basics"]], "Features": [[13, "features"]], "Main Demo": [[14, "Main-Demo"]], "Loading and Running Models": [[14, "Loading-and-Running-Models"]], "Example: Logit Lens": [[14, "Example:-Logit-Lens"]], "Intervening during a forward pass": [[14, "Intervening-during-a-forward-pass"]], "Cross-Prompt Intervention": [[15, "Cross-Prompt-Intervention"]], "Multiple Token Generation": [[16, "Multiple-Token-Generation"]], "Token Based Indexing": [[17, "Token-Based-Indexing"]]}, "indexentries": {"invoker (class in nnsight.contexts.invoker)": [[2, "nnsight.contexts.Invoker.Invoker"]], "runner (class in nnsight.contexts.runner)": [[2, "nnsight.contexts.Runner.Runner"]], "tracer (class in nnsight.contexts.tracer)": [[2, "nnsight.contexts.Tracer.Tracer"]], "args (nnsight.contexts.invoker.invoker attribute)": [[2, "nnsight.contexts.Invoker.Invoker.args"]], "args (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.args"]], "batch_size (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.batch_size"]], "batched_input (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.batched_input"]], "blocking (nnsight.contexts.runner.runner attribute)": [[2, "nnsight.contexts.Runner.Runner.blocking"]], "generation (nnsight.contexts.runner.runner attribute)": [[2, "nnsight.contexts.Runner.Runner.generation"]], "generation_idx (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.generation_idx"]], "graph (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.graph"]], "input (nnsight.contexts.invoker.invoker attribute)": [[2, "nnsight.contexts.Invoker.Invoker.input"]], "kwargs (nnsight.contexts.invoker.invoker attribute)": [[2, "nnsight.contexts.Invoker.Invoker.kwargs"]], "kwargs (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.kwargs"]], "model (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.model"]], "module": [[2, "module-nnsight.contexts"], [2, "module-nnsight.contexts.Invoker"], [2, "module-nnsight.contexts.Runner"], [2, "module-nnsight.contexts.Tracer"], [3, "module-nnsight.intervention"], [4, "module-nnsight.models"], [4, "module-nnsight.models.AbstractModel"], [4, "module-nnsight.models.DiffuserModel"], [4, "module-nnsight.models.LanguageModel"], [5, "module-nnsight.module"], [6, "module-nnsight.patching"], [7, "module-nnsight.tracing"], [7, "module-nnsight.tracing.Graph"], [7, "module-nnsight.tracing.Node"], [7, "module-nnsight.tracing.Proxy"], [8, "module-nnsight.util"]], "next() (nnsight.contexts.invoker.invoker method)": [[2, "nnsight.contexts.Invoker.Invoker.next"]], "nnsight.contexts": [[2, "module-nnsight.contexts"]], "nnsight.contexts.invoker": [[2, "module-nnsight.contexts.Invoker"]], "nnsight.contexts.runner": [[2, "module-nnsight.contexts.Runner"]], "nnsight.contexts.tracer": [[2, "module-nnsight.contexts.Tracer"]], "output (nnsight.contexts.tracer.tracer attribute)": [[2, "nnsight.contexts.Tracer.Tracer.output"]], "remote (nnsight.contexts.runner.runner attribute)": [[2, "nnsight.contexts.Runner.Runner.remote"]], "run_local() (nnsight.contexts.runner.runner method)": [[2, "nnsight.contexts.Runner.Runner.run_local"]], "save_all() (nnsight.contexts.invoker.invoker method)": [[2, "nnsight.contexts.Invoker.Invoker.save_all"]], "scan (nnsight.contexts.invoker.invoker attribute)": [[2, "nnsight.contexts.Invoker.Invoker.scan"]], "tracer (nnsight.contexts.invoker.invoker attribute)": [[2, "nnsight.contexts.Invoker.Invoker.tracer"]], "hookmodel (class in nnsight.intervention)": [[3, "nnsight.intervention.HookModel"]], "interventionproxy (class in nnsight.intervention)": [[3, "nnsight.intervention.InterventionProxy"]], "tokenindexer (class in nnsight.intervention)": [[3, "nnsight.intervention.TokenIndexer"]], "handles (nnsight.intervention.hookmodel attribute)": [[3, "nnsight.intervention.HookModel.handles"]], "input_hook (nnsight.intervention.hookmodel attribute)": [[3, "nnsight.intervention.HookModel.input_hook"]], "intervene() (in module nnsight.intervention)": [[3, "nnsight.intervention.intervene"]], "model (nnsight.intervention.hookmodel attribute)": [[3, "nnsight.intervention.HookModel.model"]], "modules (nnsight.intervention.hookmodel attribute)": [[3, "nnsight.intervention.HookModel.modules"]], "nnsight.intervention": [[3, "module-nnsight.intervention"]], "output_hook (nnsight.intervention.hookmodel attribute)": [[3, "nnsight.intervention.HookModel.output_hook"]], "save() (nnsight.intervention.interventionproxy method)": [[3, "nnsight.intervention.InterventionProxy.save"]], "shape (nnsight.intervention.interventionproxy property)": [[3, "nnsight.intervention.InterventionProxy.shape"]], "t (nnsight.intervention.interventionproxy property)": [[3, "nnsight.intervention.InterventionProxy.t"]], "token (nnsight.intervention.interventionproxy property)": [[3, "nnsight.intervention.InterventionProxy.token"]], "value (nnsight.intervention.interventionproxy property)": [[3, "nnsight.intervention.InterventionProxy.value"]], "abstractmodel (class in nnsight.models.abstractmodel)": [[4, "nnsight.models.AbstractModel.AbstractModel"]], "diffuser (class in nnsight.models.diffusermodel)": [[4, "nnsight.models.DiffuserModel.Diffuser"]], "diffusermodel (class in nnsight.models.diffusermodel)": [[4, "nnsight.models.DiffuserModel.DiffuserModel"]], "languagemodel (class in nnsight.models.languagemodel)": [[4, "nnsight.models.LanguageModel.LanguageModel"]], "alter (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.alter"]], "args (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.args"]], "config (nnsight.models.languagemodel.languagemodel attribute)": [[4, "nnsight.models.LanguageModel.LanguageModel.config"]], "custom_model (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.custom_model"]], "dispatch (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.dispatch"]], "dispatched (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.dispatched"]], "forward() (nnsight.models.abstractmodel.abstractmodel method)": [[4, "nnsight.models.AbstractModel.AbstractModel.forward"]], "generate() (nnsight.models.abstractmodel.abstractmodel method)": [[4, "nnsight.models.AbstractModel.AbstractModel.generate"]], "invoke() (nnsight.models.abstractmodel.abstractmodel method)": [[4, "nnsight.models.AbstractModel.AbstractModel.invoke"]], "kwargs (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.kwargs"]], "local_model (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.local_model"]], "local_model (nnsight.models.languagemodel.languagemodel attribute)": [[4, "nnsight.models.LanguageModel.LanguageModel.local_model"]], "meta_model (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.meta_model"]], "meta_model (nnsight.models.languagemodel.languagemodel attribute)": [[4, "nnsight.models.LanguageModel.LanguageModel.meta_model"]], "modulize() (nnsight.models.abstractmodel.abstractmodel method)": [[4, "nnsight.models.AbstractModel.AbstractModel.modulize"]], "nnsight.models": [[4, "module-nnsight.models"]], "nnsight.models.abstractmodel": [[4, "module-nnsight.models.AbstractModel"]], "nnsight.models.diffusermodel": [[4, "module-nnsight.models.DiffuserModel"]], "nnsight.models.languagemodel": [[4, "module-nnsight.models.LanguageModel"]], "repoid_path_clsname (nnsight.models.abstractmodel.abstractmodel attribute)": [[4, "nnsight.models.AbstractModel.AbstractModel.repoid_path_clsname"]], "tokenizer (nnsight.models.languagemodel.languagemodel attribute)": [[4, "nnsight.models.LanguageModel.LanguageModel.tokenizer"]], "module (class in nnsight.module)": [[5, "nnsight.module.Module"]], "backward_input (nnsight.module.module property)": [[5, "nnsight.module.Module.backward_input"]], "backward_output (nnsight.module.module property)": [[5, "nnsight.module.Module.backward_output"]], "input (nnsight.module.module property)": [[5, "nnsight.module.Module.input"]], "input_shape (nnsight.module.module attribute)": [[5, "nnsight.module.Module.input_shape"]], "input_type (nnsight.module.module attribute)": [[5, "nnsight.module.Module.input_type"]], "module_path (nnsight.module.module attribute)": [[5, "nnsight.module.Module.module_path"]], "nnsight.module": [[5, "module-nnsight.module"]], "output (nnsight.module.module attribute)": [[5, "id0"], [5, "nnsight.module.Module.output"]], "output (nnsight.module.module property)": [[5, "id1"]], "output_shape (nnsight.module.module attribute)": [[5, "nnsight.module.Module.output_shape"]], "output_type (nnsight.module.module attribute)": [[5, "nnsight.module.Module.output_type"]], "tracer (nnsight.module.module attribute)": [[5, "nnsight.module.Module.tracer"]], "wrap() (nnsight.module.module static method)": [[5, "nnsight.module.Module.wrap"]], "patch (class in nnsight.patching)": [[6, "nnsight.patching.Patch"]], "patcher (class in nnsight.patching)": [[6, "nnsight.patching.Patcher"]], "add() (nnsight.patching.patcher method)": [[6, "nnsight.patching.Patcher.add"]], "nnsight.patching": [[6, "module-nnsight.patching"]], "obj (nnsight.patching.patch attribute)": [[6, "nnsight.patching.Patch.obj"]], "parent (nnsight.patching.patch attribute)": [[6, "nnsight.patching.Patch.parent"]], "patch() (nnsight.patching.patch method)": [[6, "nnsight.patching.Patch.patch"]], "patches (nnsight.patching.patcher attribute)": [[6, "nnsight.patching.Patcher.patches"]], "replacement (nnsight.patching.patch attribute)": [[6, "nnsight.patching.Patch.replacement"]], "restore() (nnsight.patching.patch method)": [[6, "nnsight.patching.Patch.restore"]], "graph (class in nnsight.tracing.graph)": [[7, "nnsight.tracing.Graph.Graph"]], "node (class in nnsight.tracing.node)": [[7, "nnsight.tracing.Node.Node"]], "proxy (class in nnsight.tracing.proxy)": [[7, "nnsight.tracing.Proxy.Proxy"]], "_proxy_device (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node._proxy_device"]], "add() (nnsight.tracing.graph.graph method)": [[7, "nnsight.tracing.Graph.Graph.add"]], "args (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.args"]], "argument_node_names (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.argument_node_names"]], "compile() (nnsight.tracing.graph.graph method)": [[7, "nnsight.tracing.Graph.Graph.compile"]], "compile() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.compile"]], "dependencies (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.dependencies"]], "destroy() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.destroy"]], "execute() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.execute"]], "fulfilled() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.fulfilled"]], "generation_idx (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.generation_idx"]], "graph (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.graph"]], "increment() (nnsight.tracing.graph.graph method)": [[7, "nnsight.tracing.Graph.Graph.increment"]], "kwargs (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.kwargs"]], "listeners (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.listeners"]], "meta (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.meta"]], "module_proxy (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.module_proxy"]], "name (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.name"]], "name_idx (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.name_idx"]], "nnsight.tracing": [[7, "module-nnsight.tracing"]], "nnsight.tracing.graph": [[7, "module-nnsight.tracing.Graph"]], "nnsight.tracing.node": [[7, "module-nnsight.tracing.Node"]], "nnsight.tracing.proxy": [[7, "module-nnsight.tracing.Proxy"]], "node (nnsight.tracing.proxy.proxy attribute)": [[7, "nnsight.tracing.Proxy.Proxy.node"]], "nodes (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.nodes"]], "prepare_inputs() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.prepare_inputs"]], "prepare_proxy_values() (nnsight.tracing.node.node static method)": [[7, "nnsight.tracing.Node.Node.prepare_proxy_values"]], "proxy() (nnsight.tracing.graph.graph method)": [[7, "nnsight.tracing.Graph.Graph.proxy"]], "proxy_class (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.proxy_class"]], "proxy_device (nnsight.tracing.node.node property)": [[7, "nnsight.tracing.Node.Node.proxy_device"]], "proxy_update() (nnsight.tracing.proxy.proxy static method)": [[7, "nnsight.tracing.Proxy.Proxy.proxy_update"]], "proxy_value (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.proxy_value"]], "proxy_wrapper() (in module nnsight.tracing.proxy)": [[7, "nnsight.tracing.Proxy.proxy_wrapper"]], "redundant() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.redundant"]], "rtn() (nnsight.tracing.graph.graph static method)": [[7, "nnsight.tracing.Graph.Graph.rtn"]], "set_value() (nnsight.tracing.node.node method)": [[7, "nnsight.tracing.Node.Node.set_value"]], "target (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.target"]], "trace() (nnsight.tracing.graph.graph static method)": [[7, "nnsight.tracing.Graph.Graph.trace"]], "validate (nnsight.tracing.graph.graph attribute)": [[7, "nnsight.tracing.Graph.Graph.validate"]], "value (nnsight.tracing.node.node attribute)": [[7, "nnsight.tracing.Node.Node.value"]], "wrap() (nnsight.tracing.graph.graph method)": [[7, "nnsight.tracing.Graph.Graph.wrap"]], "wrappermodule (class in nnsight.util)": [[8, "nnsight.util.WrapperModule"]], "apply() (in module nnsight.util)": [[8, "nnsight.util.apply"]], "cross_entropy_loss() (in module nnsight.util)": [[8, "nnsight.util.cross_entropy_loss"]], "fetch_attr() (in module nnsight.util)": [[8, "nnsight.util.fetch_attr"]], "forward() (nnsight.util.wrappermodule method)": [[8, "nnsight.util.WrapperModule.forward"]], "nnsight.util": [[8, "module-nnsight.util"]], "timed() (in module nnsight.util)": [[8, "nnsight.util.timed"]], "wrap() (in module nnsight.util)": [[8, "nnsight.util.wrap"]]}})
\ No newline at end of file
diff --git a/public/tutorials/notebooks/token_indexing/index.html b/public/tutorials/notebooks/token_indexing/index.html
index 9c3e894..4d7c9f8 100644
--- a/public/tutorials/notebooks/token_indexing/index.html
+++ b/public/tutorials/notebooks/token_indexing/index.html
@@ -53,7 +53,7 @@
     <link rel="author" title="About these documents" href="../../../about/" />
     <link rel="index" title="Index" href="../../../genindex/" />
     <link rel="search" title="Search" href="../../../search/" />
-    <link rel="next" title="About nnsight: a transparent science API for black-box inference" href="../../../about/" />
+    <link rel="next" title="About nnsight" href="../../../about/" />
     <link rel="prev" title="Multiple Token Generation" href="../multiple_token/" />
   <meta name="viewport" content="width=device-width, initial-scale=1"/>
   <meta name="docsearch:language" content="en"/>
@@ -582,7 +582,7 @@ <h1>Token Based Indexing<a class="headerlink" href="#Token-Based-Indexing" title
        title="next page">
       <div class="prev-next-info">
         <p class="prev-next-subtitle">next</p>
-        <p class="prev-next-title">About nnsight: a transparent science API for black-box inference</p>
+        <p class="prev-next-title">About nnsight</p>
       </div>
       <i class="fa-solid fa-angle-right"></i>
     </a>