Skip to content

Commit

Permalink
fix: code copy button (#20)
Browse files Browse the repository at this point in the history
  • Loading branch information
michele-milesi authored Dec 12, 2023
1 parent f2cd9fd commit c05354d
Show file tree
Hide file tree
Showing 5 changed files with 22 additions and 3 deletions.
2 changes: 2 additions & 0 deletions _posts/2023-05-16-functionality-checks.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ subclass: 'post'
---

# Available Functionalities
<div markdown="block" class="with-new-line">

```python

Expand Down Expand Up @@ -78,6 +79,7 @@ trainer = pl.Trainer(gpus=4, num_nodes=8, precision=16, limit_train_batches=0.5)
trainer.fit(model, train_loader, val_loader)

```
</div>

# Latex Formulas

Expand Down
4 changes: 2 additions & 2 deletions _posts/2023-05-17-welcome.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ Picture this: Within a mere five minutes, you'll have your first agent trained a

<div class="two-columns-container code-flex" style="gap: 15px; margin:15px">
<div>
<div class="code-block" style="opacity: 1">
<code class="language-bash with-new-line" data-lang="bash" style="opacity: 1; color: #f8f8f2; height: 250px">
<div class="code-block with-new-line" style="opacity: 1">
<code class="language-bash" data-lang="bash" style="opacity: 1; color: #f8f8f2; height: 250px">
git clone https://github.com/Eclectic-Sheep/sheeprl.git
cd sheeprl
python3.10 -m venv .venv
Expand Down
4 changes: 4 additions & 0 deletions _posts/2023-07-06-dreamer_v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ Our PyTorch implementation aims to be a simple, scalable and well-documented rep

As an example, the implementation of the *KL balancing* directly follows the equation above:

<div markdown="block" class="with-new-line">

```python
from torch.distributions import Independent, OneHotCategoricalStraightThrough

Expand All @@ -108,6 +110,8 @@ rhs = kl_divergence(
kl_loss = alpha * lhs + (1 - alpha) * rhs
```

</div>

Do you want to know more about how we implemented Dreamer-V2? Check out [our implementation](https://github.com/Eclectic-Sheep/sheeprl/tree/main/sheeprl/algos/dreamer_v2){:target="_blank"}.

### References
Expand Down
4 changes: 4 additions & 0 deletions _posts/2023-08-10-dreamer_v3.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ $$
#### Uniform Mix
To prevent spikes in the KL loss, the categorical distributions (the one for discrete actions and the one for the posteriors/priors) are parametrized as mixtures of $1\%$ uniform and $99\%$ neural network output. This avoid the distributions to become near deterministic. To implement the *uniform mix*, we applied the *uniform mix* function to the logits returned by the neural networks.

<div markdown="block" class="with-new-line">>

```python
import torch
from torch import Tensor
Expand All @@ -86,6 +88,8 @@ def uniform_mix(self, logits: Tensor, unimix: float = 0.01) -> Tensor:
return logits
```

</div>

#### Return regularizer for the policy
The main difficulty in Dreamer-V2 *actor learning phase* is the choosing of the entropy regularizer, which heavily depends on the scale and the frequency of the rewards. To have a single entropy coefficient, it is necessary to normalize the returns using moving statistics. In particular, they found out that it is more convenient to scale down large rewards and not scale up small rewards, to avoid adding noise.

Expand Down
11 changes: 10 additions & 1 deletion assets/js/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@ $(document).ready(function () {
});

// Document Ctrl + C
const sources = document.querySelectorAll("code:not(.with-new-line)");
const sources = document.querySelectorAll(":not(.with-new-line) code");
const sources_new_line = document.querySelectorAll(".with-new-line code");

sources.forEach(source => {
source.addEventListener("copy", (event) => {
Expand All @@ -58,4 +59,12 @@ $(document).ready(function () {
event.preventDefault();
});
});

sources_new_line.forEach(source => {
source.addEventListener("copy", (event) => {
const selection = document.getSelection();
event.clipboardData.setData("text/plain", selection.toString());
event.preventDefault();
});
});
});

0 comments on commit c05354d

Please sign in to comment.