This repository has been archived by the owner on Feb 12, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 14
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Usermanagement documentation (Nifi) + Tensorflow use case (#130)
* group management * Tensorflow use case. * NiFi - LDAP Documentation Co-authored-by: Amen Ayadi <[email protected]> Co-authored-by: Alexandre Nuttinck <[email protected]> Co-authored-by: Sebastien Dupont <[email protected]>
- Loading branch information
1 parent
1a72f18
commit b383776
Showing
18 changed files
with
227 additions
and
17 deletions.
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,101 @@ | ||
# Tensorflow simple use case | ||
|
||
This is a simple [Tensorflow](https://www.tensorflow.org/) and [Jupyter](https://jupyter.readthedocs.io/en/latest/) use case with FADI for image classification using the [MNIST dataset](http://yann.lecun.com/exdb/mnist/). | ||
|
||
|
||
Before starting, change the environment: | ||
* Click on `Control panel` | ||
* Click on `Stop my server` | ||
* Finally, click on `Start server`, choose `tensorflow environment` and click on `Spawn`. | ||
|
||
![Jupyter web interface](./images/Tensorflow.png) | ||
|
||
|
||
* Import TensorFlow: | ||
|
||
```python | ||
from __future__ import absolute_import, division, print_function, unicode_literals | ||
|
||
|
||
import tensorflow as tf | ||
``` | ||
* Load and prepare the [MNIST](http://yann.lecun.com/exdb/mnist/) dataset. Convert the samples from integers to floating-point numbers: | ||
|
||
```python | ||
mnist = tf.keras.datasets.mnist | ||
|
||
(x_train, y_train), (x_test, y_test) = mnist.load_data() | ||
x_train, x_test = x_train / 255.0, x_test / 255.0 | ||
``` | ||
* Build the `tf.keras.Sequential` model by stacking layers. Choose an optimizer and loss function for training: | ||
|
||
```python | ||
model = tf.keras.models.Sequential([ | ||
tf.keras.layers.Flatten(input_shape=(28, 28)), | ||
tf.keras.layers.Dense(128, activation='relu'), | ||
tf.keras.layers.Dropout(0.2), | ||
tf.keras.layers.Dense(10) | ||
]) | ||
``` | ||
* For each example the model returns a vector of "[logits](https://developers.google.com/machine-learning/glossary#logits)" or "[log-odds](https://developers.google.com/machine-learning/glossary#log-odds)" scores, one for each class.: | ||
|
||
```python | ||
predictions = model(x_train[:1]).numpy() | ||
predictions | ||
``` | ||
![Jupyter web interface](./images/Tensorflowusecase.png) | ||
|
||
* The `tf.nn.softmax` function converts these logits to "probabilities" for each class: | ||
|
||
```python | ||
tf.nn.softmax(predictions).numpy() | ||
``` | ||
![Jupyter web interface](./images/tensor2.png) | ||
|
||
* The `losses.SparseCategoricalCrossentropy` loss takes a vector of logits and a True index and returns a scalar loss for each example. | ||
|
||
```python | ||
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True) | ||
``` | ||
* This loss is equal to the negative log probability of the the true class: It is zero if the model is sure of the correct class. | ||
|
||
This untrained model gives probabilities close to random (1/10 for each class), so the initial loss should be close to -tf.log(1/10) ~= 2.3. | ||
|
||
```python | ||
loss_fn(y_train[:1], predictions).numpy() | ||
``` | ||
![Jupyter web interface](./images/tensor3.png) | ||
|
||
```python | ||
model.compile(optimizer='adam', | ||
loss=loss_fn, | ||
metrics=['accuracy']) | ||
``` | ||
* The `Model.fit` method adjusts the model parameters to minimize the loss: | ||
|
||
```python | ||
model.fit(x_train, y_train, epochs=5) | ||
``` | ||
|
||
![Jupyter web interface](./images/tensor4.png) | ||
|
||
* The `Model.evaluate` method checks the models performance, usually on a "[Validation-set](https://developers.google.com/machine-learning/glossary#validation-set)". | ||
|
||
```python | ||
model.evaluate(x_test, y_test, verbose=2) | ||
``` | ||
![Jupyter web interface](./images/tensor5.png) | ||
|
||
* The image classifier is now trained to ~98% accuracy on this dataset, If you want your model to return a probability, you can wrap the trained model, and attach the softmax to it: | ||
|
||
```python | ||
probability_model = tf.keras.Sequential([ | ||
model, | ||
tf.keras.layers.Softmax() | ||
]) | ||
``` | ||
|
||
```python | ||
probability_model(x_test[:5]) | ||
``` | ||
![Jupyter web interface](./images/tensor6.png) |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.