forked from alessandro308/ICT-infrastructure
-
Notifications
You must be signed in to change notification settings - Fork 6
/
index.html
2267 lines (2071 loc) · 184 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html><html><head>
<title>README</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.css">
<style>
h1, h2, h3, h4, h5, h6{
color: #159957 !important;
}
p {
text-align: justify;
}
p[align="center"]{
text-align: center;
}
/**
* prism.js Github theme based on GitHub's theme.
* @author Sam Clarke
*/
code[class*="language-"],
pre[class*="language-"] {
color: #333;
background: none;
font-family: Consolas, "Liberation Mono", Menlo, Courier, monospace;
text-align: left;
white-space: pre;
word-spacing: normal;
word-break: normal;
word-wrap: normal;
line-height: 1.4;
-moz-tab-size: 8;
-o-tab-size: 8;
tab-size: 8;
-webkit-hyphens: none;
-moz-hyphens: none;
-ms-hyphens: none;
hyphens: none;
}
/* Code blocks */
pre[class*="language-"] {
padding: .8em;
overflow: auto;
/* border: 1px solid #ddd; */
border-radius: 3px;
/* background: #fff; */
background: #f5f5f5;
}
/* Inline code */
:not(pre) > code[class*="language-"] {
padding: .1em;
border-radius: .3em;
white-space: normal;
background: #f5f5f5;
}
.token.comment,
.token.blockquote {
color: #969896;
}
.token.cdata {
color: #183691;
}
.token.doctype,
.token.punctuation,
.token.variable,
.token.macro.property {
color: #333;
}
.token.operator,
.token.important,
.token.keyword,
.token.rule,
.token.builtin {
color: #a71d5d;
}
.token.string,
.token.url,
.token.regex,
.token.attr-value {
color: #183691;
}
.token.property,
.token.number,
.token.boolean,
.token.entity,
.token.atrule,
.token.constant,
.token.symbol,
.token.command,
.token.code {
color: #0086b3;
}
.token.tag,
.token.selector,
.token.prolog {
color: #63a35c;
}
.token.function,
.token.namespace,
.token.pseudo-element,
.token.class,
.token.class-name,
.token.pseudo-class,
.token.id,
.token.url-reference .token.variable,
.token.attr-name {
color: #795da3;
}
.token.entity {
cursor: help;
}
.token.title,
.token.title .token.punctuation {
font-weight: bold;
color: #1d3e81;
}
.token.list {
color: #ed6a43;
}
.token.inserted {
background-color: #eaffea;
color: #55a532;
}
.token.deleted {
background-color: #ffecec;
color: #bd2c00;
}
.token.bold {
font-weight: bold;
}
.token.italic {
font-style: italic;
}
/* JSON */
.language-json .token.property {
color: #183691;
}
.language-markup .token.tag .token.punctuation {
color: #333;
}
/* CSS */
code.language-css,
.language-css .token.function {
color: #0086b3;
}
/* YAML */
.language-yaml .token.atrule {
color: #63a35c;
}
code.language-yaml {
color: #183691;
}
/* Ruby */
.language-ruby .token.function {
color: #333;
}
/* Markdown */
.language-markdown .token.url {
color: #795da3;
}
/* Makefile */
.language-makefile .token.symbol {
color: #795da3;
}
.language-makefile .token.variable {
color: #183691;
}
.language-makefile .token.builtin {
color: #0086b3;
}
/* Bash */
.language-bash .token.keyword {
color: #0086b3;
}
/* highlight */
pre[data-line] {
position: relative;
padding: 1em 0 1em 3em;
}
pre[data-line] .line-highlight-wrapper {
position: absolute;
top: 0;
left: 0;
background-color: transparent;
display: block;
width: 100%;
}
pre[data-line] .line-highlight {
position: absolute;
left: 0;
right: 0;
padding: inherit 0;
margin-top: 1em;
background: hsla(24, 20%, 50%,.08);
background: linear-gradient(to right, hsla(24, 20%, 50%,.1) 70%, hsla(24, 20%, 50%,0));
pointer-events: none;
line-height: inherit;
white-space: pre;
}
pre[data-line] .line-highlight:before,
pre[data-line] .line-highlight[data-end]:after {
content: attr(data-start);
position: absolute;
top: .4em;
left: .6em;
min-width: 1em;
padding: 0 .5em;
background-color: hsla(24, 20%, 50%,.4);
color: hsl(24, 20%, 95%);
font: bold 65%/1.5 sans-serif;
text-align: center;
vertical-align: .3em;
border-radius: 999px;
text-shadow: none;
box-shadow: 0 1px white;
}
pre[data-line] .line-highlight[data-end]:after {
content: attr(data-end);
top: auto;
bottom: .4em;
}html body{font-family:"Helvetica Neue",Helvetica,"Segoe UI",Arial,freesans,sans-serif;font-size:16px;line-height:1.6;color:#333;background-color:#fff;overflow:initial;box-sizing:border-box;word-wrap:break-word}html body>:first-child{margin-top:0}html body h1,html body h2,html body h3,html body h4,html body h5,html body h6{line-height:1.2;margin-top:1em;margin-bottom:16px;color:#000}html body h1{font-size:2.25em;font-weight:300;padding-bottom:.3em}html body h2{font-size:1.75em;font-weight:400;padding-bottom:.3em}html body h3{font-size:1.5em;font-weight:500}html body h4{font-size:1.25em;font-weight:600}html body h5{font-size:1.1em;font-weight:600}html body h6{font-size:1em;font-weight:600}html body h1,html body h2,html body h3,html body h4,html body h5{font-weight:600}html body h5{font-size:1em}html body h6{color:#5c5c5c}html body strong{color:#000}html body del{color:#5c5c5c}html body a:not([href]){color:inherit;text-decoration:none}html body a{color:#08c;text-decoration:none}html body a:hover{color:#00a3f5;text-decoration:none}html body img{max-width:100%}html body>p{margin-top:0;margin-bottom:16px;word-wrap:break-word}html body>ul,html body>ol{margin-bottom:16px}html body ul,html body ol{padding-left:2em}html body ul.no-list,html body ol.no-list{padding:0;list-style-type:none}html body ul ul,html body ul ol,html body ol ol,html body ol ul{margin-top:0;margin-bottom:0}html body li{margin-bottom:0}html body li.task-list-item{list-style:none}html body li>p{margin-top:0;margin-bottom:0}html body .task-list-item-checkbox{margin:0 .2em .25em -1.8em;vertical-align:middle}html body .task-list-item-checkbox:hover{cursor:pointer}html body blockquote{margin:16px 0;font-size:inherit;padding:0 15px;color:#5c5c5c;border-left:4px solid #d6d6d6}html body blockquote>:first-child{margin-top:0}html body blockquote>:last-child{margin-bottom:0}html body hr{height:4px;margin:32px 0;background-color:#d6d6d6;border:0 none}html body table{margin:10px 0 15px 0;border-collapse:collapse;border-spacing:0;display:block;width:100%;overflow:auto;word-break:normal;word-break:keep-all}html body table th{font-weight:bold;color:#000}html body table td,html body table th{border:1px solid #d6d6d6;padding:6px 13px}html body dl{padding:0}html body dl dt{padding:0;margin-top:16px;font-size:1em;font-style:italic;font-weight:bold}html body dl dd{padding:0 16px;margin-bottom:16px}html body code{font-family:Menlo,Monaco,Consolas,'Courier New',monospace;font-size:.85em !important;color:#000;background-color:#f0f0f0;border-radius:3px;padding:.2em 0}html body code::before,html body code::after{letter-spacing:-0.2em;content:"\00a0"}html body pre>code{padding:0;margin:0;font-size:.85em !important;word-break:normal;white-space:pre;background:transparent;border:0}html body .highlight{margin-bottom:16px}html body .highlight pre,html body pre{padding:1em;overflow:auto;font-size:.85em !important;line-height:1.45;border:#d6d6d6;border-radius:3px}html body .highlight pre{margin-bottom:0;word-break:normal}html body pre code,html body pre tt{display:inline;max-width:initial;padding:0;margin:0;overflow:initial;line-height:inherit;word-wrap:normal;background-color:transparent;border:0}html body pre code:before,html body pre tt:before,html body pre code:after,html body pre tt:after{content:normal}html body p,html body blockquote,html body ul,html body ol,html body dl,html body pre{margin-top:0;margin-bottom:16px}html body kbd{color:#000;border:1px solid #d6d6d6;border-bottom:2px solid #c7c7c7;padding:2px 4px;background-color:#f0f0f0;border-radius:3px}@media print{html body{background-color:#fff}html body h1,html body h2,html body h3,html body h4,html body h5,html body h6{color:#000;page-break-after:avoid}html body blockquote{color:#5c5c5c}html body pre{page-break-inside:avoid}html body table{display:table}html body img{display:block;max-width:100%;max-height:100%}html body pre,html body code{word-wrap:break-word;white-space:pre}}.markdown-preview{width:100%;height:100%;box-sizing:border-box}.markdown-preview .pagebreak,.markdown-preview .newpage{page-break-before:always}.markdown-preview pre.line-numbers{position:relative;padding-left:3.8em;counter-reset:linenumber}.markdown-preview pre.line-numbers>code{position:relative}.markdown-preview pre.line-numbers .line-numbers-rows{position:absolute;pointer-events:none;top:1em;font-size:100%;left:0;width:3em;letter-spacing:-1px;border-right:1px solid #999;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.markdown-preview pre.line-numbers .line-numbers-rows>span{pointer-events:none;display:block;counter-increment:linenumber}.markdown-preview pre.line-numbers .line-numbers-rows>span:before{content:counter(linenumber);color:#999;display:block;padding-right:.8em;text-align:right}.markdown-preview .mathjax-exps .MathJax_Display{text-align:center !important}.markdown-preview:not([for="preview"]) .code-chunk .btn-group{display:none}.markdown-preview:not([for="preview"]) .code-chunk .status{display:none}.markdown-preview:not([for="preview"]) .code-chunk .output-div{margin-bottom:16px}.scrollbar-style::-webkit-scrollbar{width:8px}.scrollbar-style::-webkit-scrollbar-track{border-radius:10px;background-color:transparent}.scrollbar-style::-webkit-scrollbar-thumb{border-radius:5px;background-color:rgba(150,150,150,0.66);border:4px solid rgba(150,150,150,0.66);background-clip:content-box}html body[for="html-export"]:not([data-presentation-mode]){position:relative;width:100%;height:100%;top:0;left:0;margin:0;padding:0;overflow:auto}html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{position:relative;top:0}@media screen and (min-width:914px){html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{padding:2em calc(50% - 457px + 2em)}}@media screen and (max-width:914px){html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{padding:2em}}@media screen and (max-width:450px){html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{font-size:14px !important;padding:1em}}@media print{html body[for="html-export"]:not([data-presentation-mode]) #sidebar-toc-btn{display:none}}html body[for="html-export"]:not([data-presentation-mode]) #sidebar-toc-btn{position:fixed;bottom:8px;left:8px;font-size:28px;cursor:pointer;color:inherit;z-index:99;width:32px;text-align:center;opacity:.4}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] #sidebar-toc-btn{opacity:1}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc{position:fixed;top:0;left:0;width:300px;height:100%;padding:32px 0 48px 0;font-size:14px;box-shadow:0 0 4px rgba(150,150,150,0.33);box-sizing:border-box;overflow:auto;background-color:inherit}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar{width:8px}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar-track{border-radius:10px;background-color:transparent}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar-thumb{border-radius:5px;background-color:rgba(150,150,150,0.66);border:4px solid rgba(150,150,150,0.66);background-clip:content-box}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc a{text-decoration:none}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc ul{padding:0 1.6em;margin-top:.8em}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc li{margin-bottom:.8em}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc ul{list-style-type:none}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{left:300px;width:calc(100% - 300px);padding:2em calc(50% - 457px - 150px);margin:0;box-sizing:border-box}@media screen and (max-width:1274px){html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{padding:2em}}@media screen and (max-width:450px){html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{width:100%}}html body[for="html-export"]:not([data-presentation-mode]):not([html-show-sidebar-toc]) .markdown-preview{left:50%;transform:translateX(-50%)}html body[for="html-export"]:not([data-presentation-mode]):not([html-show-sidebar-toc]) .md-sidebar-toc{display:none}
/* Please visit the URL below for more information: */
/* https://shd101wyy.github.io/markdown-preview-enhanced/#/customize-css */
</style>
</head>
<body for="html-export">
<div class="mume markdown-preview ">
<p><a href="#contributors"><img src="https://img.shields.io/badge/all_contributors-8-orange.svg?style=flat-square" alt="All Contributors"></a></p>
<p>ICT Infrastructures - University of Pisa (Italy)</p>
<p><em>Since there is only little material on ICT Infrastructures course, this is a recap and summary of classes. The notes are a compilation of the course contents and focus on the topics in accordance with Prof. Antonio Cisternino's OneNote Notebook.<br>
It is highly recommended to study with the EMC DELL slides provided under <<_Raccolta contenuto>> which will not be uploaded here for copyright reasons. Each heading correspond to a module. If you find any error please, fork and submit a pull request!</em></p>
<h1 class="mume-header" id="table-of-contents">Table of contents</h1>
<details>
<summary>Click to show or hide</summary>
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
<ul>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#cloud-computing-reference-model-module-2">Cloud Computing Reference Model [Module 2]</a></li>
<li><a href="#data-centers">Data centers</a></li>
<li><a href="#design-and-architectures">Design and Architectures</a>
<ul>
<li><a href="#cooling">Cooling</a>
<ul>
<li><a href="#crac-computer-room-air-conditioner">CRAC: Computer Room Air Conditioner</a></li>
<li><a href="#hotcold-aisles">Hot/Cold aisles</a></li>
<li><a href="#in-row-cooling">In-Row cooling</a></li>
<li><a href="#liquid-cooling">Liquid cooling</a></li>
<li><a href="#other-ideas">Other ideas</a></li>
</ul>
</li>
<li><a href="#current">Current</a>
<ul>
<li><a href="#power-distribution">Power Distribution</a>
<ul>
<li><a href="#power-factor">Power factor</a></li>
</ul>
</li>
<li><a href="#pue-power-usage-effectiveness">PUE: Power Usage Effectiveness</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#fabric">Fabric</a>
<ul>
<li><a href="#ethernet">Ethernet</a></li>
<li><a href="#infiniband">Infiniband</a></li>
<li><a href="#rdma-remote-direct-memory-access">RDMA: Remote Direct Memory Access</a></li>
<li><a href="#omni-path">Omni-Path</a></li>
<li><a href="#connectors--plugs">Connectors & plugs</a></li>
<li><a href="#software-defined-approach">Software Defined Approach</a>
<ul>
<li><a href="#sdn-software-defined-networking">SDN: Software Defined Networking</a></li>
<li><a href="#software-defined-data-center">Software-defined data center</a></li>
<li><a href="#hyper-convergence">Hyper-convergence</a></li>
</ul>
</li>
<li><a href="#network-topologies">Network topologies</a>
<ul>
<li><a href="#introduction-1">Introduction</a>
<ul>
<li><a href="#small-world-theory">Small-world theory</a></li>
<li><a href="#spanning-tree-protocol-stp">Spanning Tree Protocol (STP)</a></li>
<li><a href="#network-chassis">Network Chassis</a></li>
<li><a href="#stacking">Stacking</a></li>
</ul>
</li>
<li><a href="#three-tier-design">Three-tier design</a></li>
<li><a href="#spine-and-leaf-architecture">Spine and leaf Architecture</a></li>
<li><a href="#oversubscription">Oversubscription</a></li>
<li><a href="#some-considerations-about-numbers">Some considerations about numbers</a></li>
<li><a href="#full-fat-tree">Full Fat Tree</a></li>
<li><a href="#vlan">VLAN</a></li>
<li><a href="#switch-anatomy">Switch Anatomy</a></li>
<li><a href="#network-topology-with-firewalls">Network topology with firewalls</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#disks-and-storage">Disks and Storage</a>
<ul>
<li><a href="#interfaces">Interfaces</a></li>
<li><a href="#redundancy">Redundancy</a></li>
<li><a href="#memory-hierarchy">Memory Hierarchy</a>
<ul>
<li><a href="#nvme">NVMe</a></li>
<li><a href="#nvdimm">nvDIMM</a></li>
<li><a href="#misc">Misc</a></li>
</ul>
</li>
<li><a href="#storage-aggregation">Storage aggregation</a></li>
<li><a href="#network-area-storage-nas">Network Area Storage (NAS)</a></li>
<li><a href="#storage-area-network-san">Storage Area Network (SAN)</a></li>
<li><a href="#hci---hyperconvergent-systems">HCI - Hyperconvergent Systems</a></li>
<li><a href="#sds---software-defined-storage">SDS - Software Defined Storage</a></li>
<li><a href="#non-raid-drive-architectures">Non-RAID drive architectures</a></li>
<li><a href="#some-consideration-about-flash-drives">Some consideration about Flash Drives</a></li>
<li><a href="#storage-in-the-future">Storage in the future</a></li>
</ul>
</li>
<li><a href="#servers">Servers</a>
<ul>
<li><a href="#types-of-compute-systems">Types of compute systems</a></li>
<li><a href="#form-factors">Form-factors</a></li>
<li><a href="#misc-1">Misc</a></li>
</ul>
</li>
<li><a href="#cloud">Cloud</a>
<ul>
<li><a href="#cloud-computing-layers">Cloud computing Layers</a>
<ul>
<li><a href="#cross-functional-layers">Cross functional layers</a></li>
<li><a href="#physical-layer">Physical Layer</a></li>
<li><a href="#virtual-layer">Virtual Layer</a>
<ul>
<li><a href="#vm-network-components">VM Network components</a></li>
<li><a href="#vm-components">VM components</a></li>
<li><a href="#types-of-virtualization">Types of virtualization</a></li>
<li><a href="#virtual-machine-vm">Virtual Machine (VM)</a></li>
<li><a href="#vram-ballooning">vRAM ballooning</a></li>
<li><a href="#docker">Docker</a></li>
</ul>
</li>
<li><a href="#control-layer">Control Layer</a>
<ul>
<li><a href="#key-phases-for-provisioning-resources">Key phases for provisioning resources</a></li>
<li><a href="#thin-provisioning">Thin provisioning</a></li>
<li><a href="#open-stack">Open stack</a></li>
</ul>
</li>
<li><a href="#service-layer">Service layer</a>
<ul>
<li><a href="#service-layer-1">Service layer</a></li>
</ul>
</li>
<li><a href="#orchestration-layer">Orchestration layer</a>
<ul>
<li><a href="#cloud-portal">Cloud portal</a></li>
<li><a href="#orchestration-types">Orchestration types</a></li>
<li><a href="#orchestration-apis">Orchestration APIs</a></li>
<li><a href="#example-of-orchestration-workflows">Example of orchestration workflows</a></li>
<li><a href="#service-orchestration">Service orchestration</a></li>
</ul>
</li>
<li><a href="#business-continuity-layer">Business Continuity layer</a>
<ul>
<li><a href="#single-point-of-failure">Single point of failure</a></li>
<li><a href="#redundancy-1">Redundancy</a></li>
<li><a href="#service-availability-zones">Service Availability Zones</a></li>
<li><a href="#live-migration-of-a-vm">Live Migration of a VM</a></li>
<li><a href="#server-setup-checklist">Server Setup Checklist</a></li>
<li><a href="#backups">Backups</a></li>
</ul>
</li>
<li><a href="#security-layer--todo-complete">Security layer (TODO: complete)</a>
<ul>
<li><a href="#levels-of-security">Levels of security</a></li>
<li><a href="#firwall">Firwall</a></li>
</ul>
</li>
<li><a href="#service-managment-layer">Service Managment layer</a>
<ul>
<li><a href="#service-operation-management">Service Operation management</a></li>
<li><a href="#capacity-planning-management">Capacity Planning/ Management</a></li>
<li><a href="#incidentproblem-management">Incident/Problem Management</a></li>
<li><a href="#examples">Examples</a></li>
<li><a href="#gdpr-general-data-protection-regulation">GDPR General Data Protection Regulation</a></li>
<li><a href="#vendor-lock-in">Vendor Lock-in</a></li>
<li><a href="#standardization-portability">Standardization-Portability</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#misc-2">Misc</a></li>
</ul>
</li>
<li><a href="#in-class-exercises">In class exercises</a>
<ul>
<li><a href="#1-spine-and-leaf-vs-traditional-architecture">1) Spine and leaf VS traditional architecture</a>
<ul>
<li><a href="#question">Question</a></li>
<li><a href="#solution">Solution</a></li>
</ul>
</li>
<li><a href="#spine-and-leaf">Spine and Leaf</a></li>
<li><a href="#2-orchestration-layer">2) Orchestration layer</a>
<ul>
<li><a href="#question-1">Question</a></li>
<li><a href="#solution-1">Solution</a></li>
</ul>
</li>
<li><a href="#3-datacenter-architecture">3) Datacenter architecture</a>
<ul>
<li><a href="#question-2">Question</a></li>
<li><a href="#solution-2">Solution</a></li>
<li><a href="#solution-2">Solution 2</a></li>
</ul>
</li>
<li><a href="#4-san-vs-hyperconvergent-architecture">4) SAN VS Hyperconvergent architecture</a>
<ul>
<li><a href="#question-3">Question</a></li>
<li><a href="#solution-3">Solution</a></li>
</ul>
</li>
<li><a href="#5-dimension-a-hyperconvergent-system">5) Dimension a hyperconvergent system</a>
<ul>
<li><a href="#question-4">Question</a></li>
<li><a href="#solution-4">Solution</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#other-questions">Other questions</a>
<ul>
<li><a href="#megantosh">@megantosh</a>
<ul>
<li><a href="#solution-1">Solution 1</a></li>
<li><a href="#solution-2-1">Solution 2</a></li>
</ul>
</li>
<li><a href="#giacomodeliberali">@giacomodeliberali</a></li>
</ul>
</li>
<li><a href="#about-numbers">About numbers</a>
<ul>
<li><a href="#current-1">Current</a></li>
<li><a href="#fabric-1">Fabric</a></li>
<li><a href="#disk-and-storage">Disk and Storage</a></li>
</ul>
</li>
<li><a href="#real-use-cases">Real Use Cases</a></li>
<li><a href="#open-source">Open Source</a></li>
<li><a href="#books--guides">Books & Guides</a></li>
<li><a href="#references">References</a></li>
<li><a href="#contributors">Contributors</a></li>
</ul>
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
</details>
<h1 class="mume-header" id="introduction">Introduction</h1>
<p>The ICT world is changing (and will keep changing beyond the last time these notes were updated) and a lot of axioms about its infrastructures are becoming outdated. A couple of examples:</p>
<ul>
<li>
<p>a few years ago it was known that the main bandwidth bottleneck was the disk, and so a system's whole performance was evaluated with reference to disk usage, number of IOs operations and so on... This, nowadays, is false. Just think of <a href="https://www.anandtech.com/show/11702/intel-introduces-new-ruler-ssd-for-servers">Intel Optane SSD</a> where the new SSD technology based on 3D NAND permits to write and read faster than previous SSD, and so we have to redesign the entire system accordingly. Additionally, nvRAM (non-volatile RAM) are becoming more of an industry standard. nvRAM is a storage module similar to the hard drive but much faster.</p>
</li>
<li>
<p>In application and server distribution. In the past many applications were managed on each server with a shared storage. Nowadays we have deploy a large application on server clusters (i.e. a server node composed of multiple units working together as one) with local storage, so new system to develop and manage distributed computing application is needed (Hadoop, Cassandra, Spark...).</p>
</li>
</ul>
<h1 class="mume-header" id="cloud-computing-reference-model-module-2">Cloud Computing Reference Model [Module 2]</h1>
<p>Since the course revolves around Cloud Computing architectures, it is important to keep the following reference model of the cloud stack in mind:</p>
<p align="center">
<img width="800" src="./assets/referenceModel.png">
</p>
<ol>
<li><strong>Physical Layer [Module 3]</strong>: Foundation layer of the cloud infrastructure.<br>
The physical infrastructure supporting the operation of the cloud</li>
<li><strong>Virtual Layer [Module 4]</strong>: Abstracts physical resources and makes them appear as virtual resources. e.g. a physical server is partitioned into many virtual ones to use the hardware better. The <em>High Performance Computing</em> model bypasses the virtual layer for performance reasons.</li>
<li><strong>Control Layer [Module 5]</strong>: Dynamic Resource configuration and allocation.</li>
<li><strong>Orchestration Layer [Module 6]</strong>: workflows for task automation.</li>
<li><strong>Service Layer [Module 6]</strong>: self-service portal/interface and service catalog. Allows cloud users to obtain the resources they need without knowing where they are allocated.</li>
<li><strong>Service Management [Module 9]</strong>: on operational and business level</li>
<li><strong>Business Continuity [Module 7]</strong>: Enables ensuring the availability of services in line with SLAs.<br>
e.g. <em>Backups</em> vs <em>Replicas</em>: doing a backup of 1 PB may be a problem.<br>
<em>Fault Tolerance</em>: I should be able to power off a server without anyone noticing it.<br>
<em>live migration</em>: upgrading the software or the firmware while the system is running.</li>
<li><strong>Security [Module 8]</strong>: Governance, Risk and compliance. Also things like GDPR, phishing, antivirus, firewalls and DoS Attacks..</li>
</ol>
<h1 class="mume-header" id="data-centers">Data centers</h1>
<p>We start the course with datacenter design, see how it is built to support current and future design considerations, scalability, etc.</p>
<p>A data center is a facility used to house computer systems and associated components, such as telecommunications and storage systems. It generally includes redundant or backup components and infrastructure for power supply, data communications connections, environmental controls (e.g. air conditioning, fire suppression) and various security devices. A large data center is an industrial-scale operation using as much electricity as a small town.</p>
<p>On average there are only 6 person managing 1 million servers.<br>
Prefabricated group of racks, already cabled and cooled, are automatically inserted in the datacenter (POD - Point Of Delivery). If something is not working in the prefabricated, the specific server is shut down. If more than the 70% is not working the POD producer will simply change the entire unity.</p>
<p>The datacenter is a place where we concentrate IT system in order to reduce costs. Servers are demanding in terms of current, cooling and security.</p>
<h1 class="mume-header" id="design-and-architectures">Design and Architectures</h1>
<h2 class="mume-header" id="cooling">Cooling</h2>
<p>Today cooling is air based. Just the beginning for liquid cooling.<br>
The air pushed though the server gets a 10/15 degrees temperature augment.</p>
<h4 class="mume-header" id="crac-computer-room-air-conditioner">CRAC: Computer Room Air Conditioner</h4>
<p>Popular in the '90 (3-5KW/rack), but not very efficient in terms of energy consumption.<br>
There is a <em>floating floor</em>, under which all the cabling and the cooling is performed. The air goes up because of thermal convection where it gets caught, cooled and re-introduced.</p>
<p>Drawbacks are density (if we want to go dense this approach fails) and the absence of locality. No one is using this technique today.</p>
<p align="center">
<img width="600" src="./assets/crac.png">
</p>
<h4 class="mume-header" id="hotcold-aisles">Hot/Cold aisles</h4>
<p>The building block of this architecture are hot and cold corridors, with servers front-to-front and back-to-back; that optimize cooling efficiency.</p>
<p>The <em>workload balancing</em> may be a problem: there can be the situation where a rack is hotter than the other depending on the workload, thus is difficult to module the amount of hot and cold air. In the CRAC model the solution is pumping enough for the higher consumer, but is not possible to act only where needed. That leads waste of energy. This problem is not present in the in-row cooling technology.</p>
<p align="center">
<img width="600" src="./assets/crac1.png">
</p>
<h4 class="mume-header" id="in-row-cooling">In-Row cooling</h4>
<p>In-row cooling technology is a type of air conditioning system commonly used in data centers (15-60 kW/rack) in which the cooling unit is placed between the server cabinets in a row for offering cool air to the server equipment more effectively.</p>
<p>In-row cooling systems use a horizontal airflow pattern utilizing hot aisle/cold aisle configurations and they only occupy one-half rack of row space without any additional side clearance space. Typically, each unit is about 12 inches wide by 42 inches deep.</p>
<p>These units may be a supplement to raised-floor cooling (creating a plenum to distribute conditioned air) or may be the primary cooling source on a slab floor.</p>
<p align="center">
<img width="600" src="./assets/in-row-cooling.jpg">
</p>
<p>The in-row cooling unit draws warm exhaust air directly from the hot aisle, cools it and distributes it to the cold aisle. This ensures that inlet temperatures are steady for precise operation. Coupling the air conditioning with the heat source produces an efficient direct return air path; this is called <em>close coupled cooling</em>, which also lowers the fan energy required. In-row cooling also prevents the mixing of hot and cold air, thus increasing efficiency.</p>
<p>It's possible to give more cooling to a single rack, modulating the air needed. In front of the rack there are temperature and humidity sensors. Humidity should be avoided because can condensate because of the temperature differences and therefore conduct electricity.<br>
There are systems collecting data from the sensors and adjusting the fans. The racks are covered to separate cool air and hot air. It's also possible to optimize the datacenter cooling according to the temperature changes of the region where the datacenter is and apply "static analysis" to the datacenter location, in order to optimize resource consumption according to temperature changes. Programs are available in order to simulate airflows in datacenter in order to optimize the fans.</p>
<p>Usually every 2 racks (each 70 cm) there should be a cooling row (30 cm).</p>
<h4 class="mume-header" id="liquid-cooling">Liquid cooling</h4>
<p>It's also called CoolIT, consists in making the water flow directly onto the CPUs.<br>
Having water in a data center is a risky business, but this solution lowers the temperature for ~40%. One way of chilling the water could be pushing it down to the ground. Water Distribution System, like the Power Distribution System.</p>
<p>A lot of research has been lately invested towards oil cooling computers, particularly in the contest of High Performance Computing. This is a more secure solution because the mineral oil is not a conductor and allows to immerse everything in the oil, in order to maximize the effectiveness of the cooling. The problem of this technique is that the cables slowly pump the oil out.</p>
<h4 class="mume-header" id="other-ideas">Other ideas</h4>
<p>A typical approach to cool the air is to place chillers outside the building, or by trying geocooling, which revolves around using the cold air in depth. The main idea is to make a deep hole in the ground, and make the cables pass through it.</p>
<h2 class="mume-header" id="current">Current</h2>
<p>A 32KW datacenter is small (also if it consumes the same amount of current of 10 apartments).</p>
<p>For efficiency reasons, Datacenters use Direct current (DC) insted of Alternate current (AC): A DC power architecture contains less components, which means less heat production, hence less energy loss.
However, nowadays current is transported via AC, so it is required a conversion to DC using <em>Direct Current Transformers</em>. The Industrial current has 380 Volts in 3 phases. Also, note that Direct current is more dangerous than Alternating current.</p>
<p align="center">
<img src="http://latex.codecogs.com/gif.latex?\text{Watt}%20=%20\cos\phi\cdot%20V\cdot%20A">
</p>
<p>Where <img src="http://latex.codecogs.com/gif.latex?%5Ccos%5Cphi" alt="cosfi"> is the heat dissemination happening from conversion of AC into DC current, and it is a number <= 1. <br>It gives the efficiency of the transformation and generally it changes according to the amount of current needed (idle vs under pressure).
For example an idle server with 2 CPUs (14 cores each) consumes 140 Watts.</p>
<h3 class="mume-header" id="power-distribution">Power Distribution</h3>
<p>The amount of current allowed in a data center are the Ampere on the <em>PDU</em> (Power Distribution Unit).</p>
<p>There are one or more lines (for reliability and fault tolerance reasons) coming from different generators to the datacenter (i.e. each line 80 KW , 200 A more or less. Can use it for 6 racks 32A / rack. Maybe I will not use the whole 32 A so I can put more racks).</p>
<p>The lines are attached to an <em>UPS (Uninterruptible Power Supply/Source)</em>. It is a rack or half a rack with batteries (not enough to keep-on the servers) that in some cases can power the DC for ~20 minutes. Them are also used to prevent current oscillation. There are a <em>Control Panel</em> and a <em>Generator</em>. When the power lines fail the UPS is active between their failure and the starting of the generator and ensure a smooth transition during the energy source switching. The energy that arrives to the UPS should be divided among the servers and the switches.</p>
<p>The UPS is attached to the <em>PDU</em> (Power Distribution Unit) which is linked to the server. For redundancy reasons, a server is powered by a pair of lines, that usually are attached to two different PDU. The server uses both the lines, so that there will be continuity in case of failure of a line. In the server there are the power plugs in a row that can monitored via a web server running on the rack PDU.</p>
<p>Example of rack PDU: 2 banks, 12 plugs each, 16 A each bank, 15 KW per rack, 42 servers per rack.</p>
<h4 class="mume-header" id="power-factor">Power factor</h4>
<details>
<summary>Click to expand</summary>
<p>Alternating current (AC) supplies our buildings and equipment. AC is more efficient for power companies to deliver, but when it hits the equipment's transformers, it exhibits a characteristic known as reactance.</p>
<p>Reactance reduces the useful power (watts) available from the apparent power (volt-amperes). The ratio of these two numbers is called the power factor (PF). Therefore, the actual power formula for AC circuits is watts = volts x amps x power factor. Unfortunately, the PF is rarely stated for most equipment, but it is always a number of 1.0 or less, and about the only thing with a 1.0 PF is a light bulb.</p>
<p>For years, large UPS systems were designed based on a PF of 0.8, which meant that a 100 kVA UPS would only support 80 kW of real power load.</p>
<p>The majority of large, commercial UPS systems are now designed with a PF of 0.9. This recognizes that most of today's computing technology presents a PF of between 0.95 and 0.98 to the UPS. Some UPS systems are even designed with PFs of 1.0, which means the kVA and kW ratings are identical (100 kVA = 100 kW). However, since the IT load never presents a 1.0 PF, for these UPS systems, the actual load limit will be the kVA rating.</p>
<p>Use the hardware manufacturers' online configurations if possible. As a last resort, use the server's power supply rating -- a server with a 300-Watt power supply can never draw 800 Watts. Size the power systems based on real demand loads.</p>
<p>Dual-corded equipment adds redundancy to IT hardware, and the lines share power load. If a dual-corded server has two 300-Watt power supplies, it can still draw no more than 300 Watts in your power design, because each power supply has to be able to handle the server's full load (not including power supply efficiency calculations).</p>
<p>The other way to estimate total server power consumption is to use industry norms. Unless you're hosting high performance computing, you can probably figure groupings in three levels of density: Low density cabinets run 3.5 to 5 kW; medium density run 5 to 10 kW; high density run 10 to 15 kW. The amount of each rack type to allocate depends on your operation. Generally, data centers operate with about 50% low density cabinets, 35% medium and 15% high density.</p>
<p>If your projected average is more than 1.5 times your existing average, take a closer look at the numbers. This result is fine if you expect a significant density increase, due to new business requirements or increased virtualization onto blade servers. But if there's no apparent reason for such a density growth, re-examine your assumptions.</p>
</details>
<h3 class="mume-header" id="pue-power-usage-effectiveness">PUE: Power Usage Effectiveness</h3>
<p>PUE is a ratio that describes how efficiently a computer data center uses energy; specifically, how much energy is used by the computing equipment (in contrast to cooling and other overhead).</p>
<p>PUE is the ratio of total amount of energy used by a computer data center facility to the energy delivered to computing equipment. PUE is the inverse of data center infrastructure efficiency (DCIE).</p>
<p>As example, consider that the PUE of the university's datacenter during 2018 is less 1.2, while the average italian data center's PUE are around 2-2.5.</p>
<p>If the PUE is equal to 2 means that for each Watt used for computing, 1 Watt is used for cooling.</p>
<p align="center">
<img src="http://latex.codecogs.com/gif.latex?\text{PUE}%20=%20\dfrac{\text{total%20current}}{\text{compute%20current}}%20">
</p>
<h1 class="mume-header" id="fabric">Fabric</h1>
<p>The fabric is the interconnection between nodes inside a datacenter. We can think this level as a bunch of switch and wires.</p>
<p>We refer to North-South traffic indicating the traffic outgoing and incoming to the datacenter (internet), while we refer to East-West as the internal traffic between servers.</p>
<h2 class="mume-header" id="ethernet">Ethernet</h2>
<p>The connection can be performed with various technologies, the most famous is <strong>Ethernet</strong>, commonly used in Local Area Networks (LAN) and Wide Area Networks (WAN). Ethernet use twisted pair and optic fiber links. Ethernet as some famous features such as 48-bit MAC address and Ethernet frame format that influenced other networking protocols.</p>
<p><strong>MTU</strong> (Maximum Transfer Unit) up to 9 KB with the so called <strong>Jumbo Frames</strong>.<br>
On top of ethernet there are TCP/IP protocols (this is a standard), they introduce about 70-100 micro sec of latency.</p>
<p>The disadvantage of Ethernet is the low reliability.</p>
<h2 class="mume-header" id="infiniband">Infiniband</h2>
<p>Even if Ethernet is so famous, there are other standard to communicate. <strong>InfiniBand (IB)</strong>, by Mellanox, is another standard used in high-performance computing (HPC) that features very high throughput and very low latency (about 2 microseconds). InfiniBand is a protocol and a physical infrastructure and it can send up to 2GB messages with 16 priorities level.<br>
The <a href="https://tools.ietf.org/html/rfc4391">RFC 4391</a> specifies a method for encapsulating and transmitting IPv4/IPv6 and Address Resolution Protocol (ARP) packets over InfiniBand (IB).</p>
<p>InfiniBand transmits data in packets up to 4KB. A massage can be:</p>
<ul>
<li>a remote direct memory access read from or write to a remote node (<a href="#rdma-remote-direct-memory-access"><strong>RDMA</strong></a>)</li>
<li>a channel send or receive</li>
<li>a transaction-based operation (that can be reversed)</li>
<li>a multicast transmission</li>
<li>an atomic operation</li>
</ul>
<p>Pros:</p>
<ul>
<li>no retransmissions</li>
<li>QoS, traffic preserved, reliable</li>
</ul>
<h2 class="mume-header" id="rdma-remote-direct-memory-access">RDMA: Remote Direct Memory Access</h2>
<p>Access, a direct memory access (really!) from one computer into that of another without involving either one's OS and bypassing the CPU. This permits high-throughput and low-latency networking performing. RDMA can gain this features because is not a protocol, but is on API, hence there is no overhead.</p>
<p>RDMA supports zero-copy networking by enabling the network adapter to transfer data directly to or from application memory, eliminating the need to copy data between application memory and the data buffers in the operating system, and by bypassing TCP/IP. Such transfers require no work to be done by CPUs, caches, or context switches, and transfers continue in parallel with other system operations. When an application performs an RDMA Read or Write request, the application data is delivered directly to the network, reducing latency and enabling fast message transfer. The main use case is distributed storage.</p>
<p align="center">
<img src="./assets/rdma.jpg" width="600">
</p>
<h2 class="mume-header" id="omni-path">Omni-Path</h2>
<p>Moreover, another communication architecture that exist and is interested to see is Omni-Path. This architecture is owned by Intel and performs high-performance communication(<a href="https://en.wikipedia.org/wiki/Omni-Path">Ompni-Path Wikipedia</a>).<br>
The interest of this architecture is that Intel plans to develop technology based on that will serve as the on-ramp to exascale computing (a computing system capable of the least one exaFLOPS).</p>
<h2 class="mume-header" id="connectors-plugs">Connectors & plugs</h2>
<p>Now we try to analyse the problem from the connector point of view. The fastest wire technology available is the optic fiber. It can be divided into two categories:</p>
<ul>
<li>mono-modal (1250 nm): expensive, lower loss, covers distances up to 60KM. Used in WAN/MAN</li>
<li>multi-modal (850 nm): cheap, covers distances up to 2KM. Used in datacenters</li>
</ul>
<p>They also have different transceiver. There are two kind of connectors:</p>
<ul>
<li>LC: ok for datacenters</li>
<li>SC: usually used in metropolitan areas because it has a better signal propagation</li>
</ul>
<p>There can be a cable with a LC in one side and a SC on the other side.</p>
<p>Of course, a wire is a wire, and we need something to connect it to somewhere (transceiver):</p>
<ul>
<li><strong>SPF</strong> (Small form-factor pluggable), a compact, hot-pluggable optical module transceiver
<ul>
<li>1 Gbps</li>
</ul>
</li>
<li><strong>SFP+</strong>, can be combined with some other SFP+
<ul>
<li>10 Gbps</li>
</ul>
</li>
<li><strong>QSFP</strong> (Quad SPF)
<ul>
<li>4x10 Gbps (if combined with SPF+)</li>
</ul>
</li>
<li><strong>SFP28</strong>, where the number 28 is the number of pins
<ul>
<li>25 GBps</li>
</ul>
</li>
<li><strong>QSFP28</strong> (Quad SPF28)
<ul>
<li>4x25 Gbps (if combined with SFP28)</li>
</ul>
</li>
<li><strong>RJ45</strong>, in datacenters there are almost no installations of it
<ul>
<li>10/100 Mbps, 1/2.5/5 Gbps.</li>
<li>Different cables have categories (cat4, cat5, cat6)
<ul>
<li>2.5/5 Gbps are new standards working on cat5 and cat6 cables respectively, in order to deliver more bandwidth to the WiFi access point.</li>
</ul>
</li>
</ul>
</li>
</ul>
<table>
<thead>
<tr>
<th style="text-align:center">RJ45</th>
<th style="text-align:center">SPF+</th>
<th style="text-align:center">QSPF+ transceiver module</th>
<th style="text-align:center">LC connector</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:center"><img width="100" src="./assets/rj45.jpeg"></td>
<td style="text-align:center"><img width="150" src="./assets/sfpplus.jpg"></td>
<td style="text-align:center"><img width="250" src="./assets/qsfpplus.png"></td>
<td style="text-align:center"><img width="150" src="./assets/lc-duplex.jpg"></td>
</tr>
</tbody>
</table>
<p>Nowadays we have:</p>
<ul>
<li>25 Gbps</li>
<li>50 Gbps (2 * 25)</li>
<li>100 Gbps (4 * 25)</li>
</ul>
<p>The <strong>transceiver module</strong> can serve copper or optical fiber; it has a microchip inside and is not cheap.</p>
<h2 class="mume-header" id="software-defined-approach">Software Defined Approach</h2>
<p>The Software Defined Approach, where approach is Networking (<a href="#sdn-software-defined-networking"><strong>SDN</strong></a>) or Storage (<a href="#sds---software-defined-storage"><strong>SDS</strong></a>), is a novel approach to cloud computing.</p>
<p>Software-defined approach abstracts all the infrastructure components (compute, storage, and network), and pools them into aggregated capacity. It separates the control or management functions from the underlying components to the external software, which takes over the control operations to manage the multi-vendor infrastructure components centrally.<br>
This decoupling enable to centralize all data provisioning and management tasks through software, external to the infrastructure components.<br>
The software runs on a centralized compute system or a standalone device, called the software-defined controller.</p>
<p>Benefits of software-defined approach:</p>
<ul>
<li>Improves business agility: minimizes resource provisioning time to get new services up and running</li>
<li>Provides cost efficiency: enables to effectively use the existing infrastructure and low-cost commodity hardware to lower CAPEX</li>
<li>Enables to achieve scale-out architecture</li>
<li>Provides a central point of access to all management functions</li>
</ul>
<h3 class="mume-header" id="sdn-software-defined-networking">SDN: Software Defined Networking</h3>
<p>SDN is an architecture purposing to be dynamic, manageablea and cost-effective (<a href="https://en.wikipedia.org/wiki/Software-defined_networking#Concept">SDN Wikipedia</a>). This type of software create a virtual network to manage the network with more simplicity.</p>
<p>The main concept are the following:</p>
<ul>
<li>Network control is directly programmable (also from remote)</li>
<li>The infrastructure is agile, since it can be dynamically adjustable</li>
<li>It is programmatically configured and is managed by a software-based SDN controller</li>
<li>It is Open Standard-based and Vendor-neutral</li>
</ul>
<p>There is a <strong>flow table</strong> in the switches that remembers the connection. The routing policies are adopted according to this table.<br>
Deep packet inspection made by a level 7 firewall. The firewall validates the flow and if it's aware that the flow needs bandwidth, the firewall allows it to bypass the redirection (of the firewall).</p>
<h3 class="mume-header" id="software-defined-data-center">Software-defined data center</h3>
<p>Software-defined data center is a sort of upgrade of the previous term and indicate a series of virtualization concepts such as abstraction, pooling and automation to all data center resources and services to achieve IT as a service.</p>
<h3 class="mume-header" id="hyper-convergence">Hyper-convergence</h3>
<p>So we virtualize the networking, the storage, the data center... and the cloud! Some tools, as <a href="https://www.nutanix.com/hyperconverged-infrastructure/">Nutanix</a> build the <a href="https://en.wikipedia.org/wiki/Hyper-converged_infrastructure">hyper-converged infrastructure HCI</a> technology.</p>
<p>Hyper-converged infrastructure combines common datacenter hardware using locally attached storage resources with intelligent software to create flexible building blocks that replace legacy infrastructure consisting of separate servers, storage networks, and storage arrays.</p>
<h2 class="mume-header" id="network-topologies">Network topologies</h2>
<p>A way of cabling allowing multiple computers to communicate. It's not necessary a graph,but for the reliability purpose it often realized as a set of connected nodes. At least 10% of nodes should be connected in order to guarantee a sufficient reliability (<a href="#small-world-theory">Small World Theory</a>).</p>
<p>At layer 2 there is no routing table (<em>broadcast domain</em>), even if there are some cache mechanism. The topology is more like a tree than a graph because some edges can be cutted preserving reachability and lowering the costs. In the layer 2 topology computers talk each other, for that reason there is no scalability.<br>
The layer 2 topology is widely used for broadcasting.</p>
<p>At layer 3 there are routing tables, them are keep updated by a third part, the router. The L3 topology is the mainly used for point-to-point communication.</p>
<p>In switches there are routing tables but them are used just for cache, switches working also without routing tables.</p>
<h3 class="mume-header" id="introduction-1">Introduction</h3>
<h4 class="mume-header" id="small-world-theory">Small-world theory</h4>
<p>This <a href="https://en.wikipedia.org/wiki/Small-world_network">theory</a>, formulated by Watts and Strogatz, claims that 6 hops connect us with every person in the world.<br>
According to their studies, taken two people x and y respectively strangers, x can send a message to y just asking to his acquaintances to pass the message to someone closer to y. Hop by hop, the message reaches y going only through friends of friends. On average, this operation needs only 6 steps.</p>
<p>For this reason, a good network topology should take 6 hops on average to connect 2 machines.<br>
Actually, topologically we got more than 6 hops, but adding 10% of random links across the graph the hops number easly collapse to 6.</p>
<h4 class="mume-header" id="spanning-tree-protocol-stp">Spanning Tree Protocol (STP)</h4>
<p>First of all it is necessary to understand the loop problem. A loop is a cycle of the links between various nodes which creates a "DDoS-like" situation by flooding the network.<br>
The spanning Tree Protocol is a network protocol that builds a logical loop-free topology for Ethernet networks. Taken a node as root, it builds a spanning tree from the existing topology graph, and disables all the arch that are not in use. The graph is now totally converted into a tree.</p>
<p>In networking the spanning tree is built using some Bridge Protocol Data Units (BPDUs) packages.<br>
In 2001 the IEEE introduced Rapid Spanning Tree Protocol (RSTP) that provides significantly faster spanning tree convergence after a topology change.</p>
<p>The advantage of the Spanning Tree protocol is that unplugging a link the network will autofix in less than a minute, rebuilding a new tree with the edges previously discarded. However, nowadays it is used only in campus and not in datacenters, due to its high latency of convergence (up to 10-15 seconds to activate a backup line) that is not sufficient for an always-on system.</p>
<h4 class="mume-header" id="network-chassis">Network Chassis</h4>
<p>The Network Chassis is a sort of big modular and resilient switch. At the bottom it has a pair of power plugs and then it's made of modular <strong>line cards</strong> (with some kind of ports) and a pair of <strong>RPM</strong> Routing Processing Modules (for redundancy) to ensure that the line cards work. The chassis can be over provisioned to resist to aging but it has a limit.</p>
<p align="center">
<img width="300" src="./assets/cisco-chassis.png">
</p>
<p>Pros</p>
<ul>
<li>resilient</li>
<li>1 CLI per switch</li>
<li>expandable</li>
</ul>
<p>Cons</p>
<ul>
<li>expensive</li>
<li>not entirely future proof (today some switches may need up to 1KW power supply, while years ago they needed only 200 W)</li>
<li>aging problem</li>
</ul>
<p>The chassis is connected with the rack's <strong>tor</strong> and <strong>bor</strong> (top/bottom of rack) switches via a double link.</p>
<h4 class="mume-header" id="stacking">Stacking</h4>
<p>Some network switches have the ability to be connected to other switches and operate together as a single unit. These configurations are called stacks, and are useful for quickly increasing the capacity of a network.</p>
<p>It's cheaper than the chassis but there is less redundancy and it is not upgradable without connectivity.</p>
<h3 class="mume-header" id="three-tier-design">Three-tier design</h3>
<p>Simple architecture constisting of core, aggregation and access switches connected in a hierarchy through pathways. Possible loops in those paths are prevented using the Spanning Tree Protocol, which also provides active-passive redundancy: indeed the STP tree keeps only a set of active nodes.</p>
<p>However, this type of redundancy leads to inefficient east-west traffic, because devices connected to the same switch port may contend for bandwidth.
Moreover, communication server-to-server might requires long crossings between layers, causing latency and traffic bottlenecks.
Hence, the Three-tier design is not good for virtualization, because VMs should be able to freely move between servers without compromises</p>
<h3 class="mume-header" id="spine-and-leaf-architecture">Spine and leaf Architecture</h3>
<p align="center">
<img width="800" src="./assets/spine-leaf-vs-3-tier.png">
<img width="800" src="./assets/spine-and-leaves.jpg">
</p>
<p>Architecture suitable for large datacenters and cloud networks due to its scalability, reliability and better performance.
It consists of two layers: the spine layer, which is made of switches that perform routing and that work as the backbone of the network, and the leaf layer, which is made of switches
that connect to endpoints such as servers, storage devices, firewalls, load balancers and edge routers.
Every leaf switch is interconnected to every spine switch of the network fabric. Using this topology, any server can communicate with any other server with no more than one interconnection switch path between any two leaf switches.</p>
<p>It is highly scalable: if the bandwidth is not enough, simply add an additional spine switch and connect it to all the leaf switches (it also reduces <strong>oversubscription</strong>, which is described next section);
if the ports are not enough, simply add a new leaf switch and connect it to all the spine switches </p>
<p>Loops are prevented using the <strong>Link Aggregation Control Protocol (LACP)</strong>: it aggregates two different physical links between two devices into a logical point-to-point link.
That means that both links can be used to communicate, increasing the bandwidth and gaining active-active redundancy in case of failure of a link (ensuring no loops because each link is a single channel).
Hence, the leaf-spine design provides a more stable and reliable network performance.
<p align="center">
<img src="./assets/LACP.jpg">
</p>
LACP also provides a method to control the <strong>bundling of several physical ports together to form a single logical channel</strong>. The <strong>first two ports of every switch are reserved</strong> to create a link with a twin switch (a loop is created, but the OS is aware of that and it avoids it). Next ports are the ones used to create links with leaf nodes. The <strong>bandwidth is aggregated</strong> (i.e. 2*25 Gbps), but it's still capped to 25 Gbps because the <strong>traffic goes only from one way to the other</strong> each time.</p>
<p>Usually in a spine and leaf architecture the NS traffic, that connect the datacenter to Internet, is slow and the EW traffic, that is server-to-server and rack-to-rack, is very intensive.</p>
<p>Characteristics:</p>
<ul>
<li>fixed form factor (non modular switches)</li>
<li>active-active redundancy</li>
<li>loop aware topology (a tree topology with no links disabled for redundancy reasons).</li>
<li>interconnect using standard cables (decide how many links use to interconnect spines with leaves and how many others link to racks).</li>
</ul>
<p>With this architecture it's possible to turn off one switch, upgrade it and reboot it without compromising the network. Half of the bandwidth is lost in the process, but the twin switch keeps the connection alive.</p>
<p>A typical configuration of the ports and bandwidth of the leaves is:</p>
<ul>
<li>1/3 going upwards and 2/3 going downwards</li>
<li>48 ports 10 Gbps each (downward - from leaves to racks)
<ul>
<li>plus 6 ports 40 Gbps each (upward - from leaves to spines)</li>
</ul>
</li>
<li>or 48 ports 25 each (downward)
<ul>
<li>plus 6 ports 100 each (upward)</li>
</ul>
</li>
</ul>
<p align="center">
<img width="400" src="./assets/sal-switch.png">
</p>
<p>Just a small remark: with spine and leaf we introduce <strong>more hops</strong>, so more latency, than the chassis approach. The solution for this problem is using as a base of the spine a <strong>huge switch (256 ports)</strong> which actually acts as a chassis, in order to reduce the number of hops and latency.</p>
<h3 class="mume-header" id="oversubscription">Oversubscription</h3>
<p>It is the practice of connecting multiple devices to the same switch port to optimize use. For example, it is particulary useful to connect multiple slower devices to a single port to take advantage of the unused capacity of the port and improve its utilization.
However, devices and applications that require high bandwidth should generally connect with a switch port 1-on-1, because multiple devices connected to the same switch port may contend for that port's bandwidth, resulting in poor response time.
Hence, significant increases in the use of multi-core CPUs, server virtualization, flash storage, Big Data and cloud computing have driven the requirement for modern networks to have lower oversubscription.
For this reason, it is important to keep in mind the <strong>oversubscription ratio</strong>, when designing your fabric. </p>
<p>In a leaf-spine design, this oversubscription is measured as the <strong>ratio of downlink ports</strong> (to servers/storage) <strong>to uplink ports</strong> (to spine switches).
Current <strong>modern network designs</strong> have oversubscription ratios of <strong>3:1</strong> or less.
For example, if you have 20 servers each connected with 10Gbps downlinks (leaft switches - servers) and 4 10Gbps uplinks (leaf switches - spine switches), you have a 5:1 oversubscription ratio (200Gbps/40Gbps). </p>
<p>Is it possible to achieve a degree of oversubscription equal to 1?<br>
Yes, and it is possible by just linking half the ports upwards and half down. This is the basis of the full fat tree.</p>
<h3 class="mume-header" id="some-considerations-about-numbers">Some considerations about numbers</h3>
<details>
<summary>
Click to show or hide
</summary>
Start think about real world. We have some server with 1 Gbps (not so high speed, just think that is the speed you can reach with your laptop attaching a cable that is in classroom in the university). We have to connect this servers to each other, using switches (each of them has 48 ports). We have a lots of servers... The computation is done.
<p align="center">
<img width="600" src="./assets/speed-required.png">
</p>
<p>As we see we need a lots of bandwidth to manage a lots of service and even if the north-south traffic (the traffic that goes outside from our datacenter) can be relatively small (the university connection exits on the world with 40 Gbps), the east-west traffic (the traffic inside the datacenter) can reach a very huge number of Gbps. <a href="https://www.arubacloud.com/infrastructures/italy-dc-it1.aspx">Aruba datacenter</a> (called IT1) with another Aruba datacenter (IT2) reach a bandwidth of 82 Gbps of Internet connection.</p>
</details>
<h3 class="mume-header" id="full-fat-tree">Full Fat Tree</h3>
<p>In this network topology, the link that are nearer the top of the hierarchy are "fatter" (thicker, which means high-bandwidth) than the link further down the hierarchy. <strong>Used only in high performance computing</strong> where performances have priority over budgets.</p>
<p>The full fat tree <strong>resolves the problem of over-subscription</strong>. Adopting the spine and leaf there is the risk that the links closer to the spines can't sustain the traffic coming from all the links going from the servers to the leaves. The full fat tree is a way to build a tree so that the capacity is never less than the incoming traffic. It's <strong>quite expensive</strong> and because of this reason some over subscription can be accepted.</p>
<p align="center">
<img width="600" src="./assets/full-fat-tree-network.png">
</p>
<h3 class="mume-header" id="vlan">VLAN</h3>
<p>Now, the problem is that every switch can be connected to each other and so there is no more LANs separation in the datacenter, every packet can go wherever it wants and some problems may appear. VLANs solve this problem partitioning a broadcast domain and creating isolated computer networks.</p>
<p>A virtual LAN (VLAN) is a virtual network consisting of virtual and/or physical switches, which <strong>divides a LAN into smaller logical segments</strong>. A VLAN groups the nodes with a common set of functional requirements, independent of the physical location of the nodes. In a multi-tenant cloud environment, the provider typically creates and assigns a <strong>separate VLAN to each consumer</strong>. This provides a private network and IP address space to a consumer, and ensures isolation from the network traffic of other consumers.</p>
<p>It works by applying <strong>tags</strong> (from 1 to 4094) to network packets (in Ethernet frame) and handling these tags in the networking systems.</p>
<p align="center">
<img width="600" src="./assets/vlan.png">
</p>
<p>A <strong>switch can be configured to accept some tags</strong> on some ports and some other tags on some other ports.</p>
<p>VLAN are useful to manage the <strong>access control</strong> to some resources (and avoid to access to some subnetwork from other subnetwork). Different VLANs are usually used for different purposes.</p>
<h3 class="mume-header" id="switch-anatomy">Switch Anatomy</h3>
<p>A switch is an ASIC (Application-Specific Integrated Circuit). It can be proprietary architecture or non-proprietary. There are two type of switches: Layer 2 and Layer 3 switches. The main difference is the routing function: A Layer 2 switch only deals with MAC addresses, while a Layer 3 switch also cares about IP addresses and manages VLAN and Intra-VLAN communications.
In both layers there is no loop problem.</p>
<p>Datacenter's switches are usually <strong>non-blocking</strong>. It basically means that this switches have the forwarding capacity that supports concurrently <strong>all ports at full capacity</strong>.</p>
<p>Now some standard are trying to impose a common structure to the network elements (switch included) to facilitate the creation of standard orchestration and automation tools.</p>
<p>The internal is made of a <strong>control plane</strong> which is configurable and a <strong>data plane</strong> where there are the ports and where the actual switching is made. The control plane evolved during the years, now they run an OS and Intel CPU's. Through a CLI Command Line Interface it's possible to configure the control plane. Some examples of command are:</p>
<ul>
<li>show running config</li>
<li>show interfaces status</li>
<li>show vlan</li>
<li>config (to enter in config mode)</li>
</ul>
<p>Some protocols in the switch (bold ones are important):</p>
<ul>
<li>PING to test connectivity.</li>
<li>LLDP Local Link Discovery Protocol ( a way to explore the graph).</li>
<li><strong>STP</strong> Spanning Tree Protocol (to avoid loops).</li>
<li>RSTP Rapid-STP</li>
<li>DCBX Data Center Bridging eExchange (QoS, priority)</li>
<li>PFC Priority Flow Control</li>
<li>ETS Enhanced Transmission Selection (priority)</li>
<li><strong>LACP</strong> Link Aggregation Control Protocol (use two wires as they are one).</li>
</ul>
<p><strong>ONIE</strong> (Open Networking Installed Environment) boot loader<br>
The switch has a firmware and two slots for the OS images. When updating in the first slot we store the old OS image, in the second slot the new one.</p>
<p><strong>NFV</strong> Network Functions Virtualization (5G mostly NFV based)<br>
The data plane is connected to a DC's VM which acts as a control plane.</p>
<h3 class="mume-header" id="network-topology-with-firewalls">Network topology with firewalls</h3>
<p>A Firewall can only perform security check on a flow, but cannot manage the flow itself. Furthermore, is not possible to let pass the entire traffic through the Firewall, because it would be a bottleneck. For that reason, after the security checks the firewall divert the flow directly to router and switches thanks to <a href="https://ryu.readthedocs.io/en/latest/ofproto_ref.html">OpenFlow API</a>.</p>
<h1 class="mume-header" id="disks-and-storage">Disks and Storage</h1>
<p><strong>IOPS</strong>: Input/output operations per second is an input/output performance measurement used to characterize computer storage devices (associated with an access pattern: random or sequential).</p>
<h2 class="mume-header" id="interfaces">System Bus Interfaces</h2>
<ul>
<li>SATA: with controller, slow because it is the bottleneck</li>
<li>SAS (Serial Attached SCSI)</li>
<li>NVMe (Non Volatile Memory express): controller-less, protocol used over PCI express bus</li>
<li>...</li>
</ul>
<h2 class="mume-header" id="redundancy">Redundancy</h2>
<p><a href="https://en.wikipedia.org/wiki/RAID#Standard_levels">RAID</a> stands for Redundant Array of Independent Disks. The RAID is done by the disk controller or the OS.<br>
The more common RAID configurations are:</p>
<ul>
<li>RAID-0: striping, two drivers aggregated that works as a single one (no fault tolerance)</li>
<li>RAID-1: mirroring, write on both the drives, one is the copy of the other.</li>
<li>RAID-5: block-level striping with distributed parity. It's xor based: the first bit goes in the first disk, the second bit in the second one and their xor in the third. If one disk crashes I can recompute its content with the other two (for each two bits of info I need one extra bit, so one third more disk storage). This means mirroring with only 50% more space.</li>
<li>RAID-6: block-level striping with double distributed parity. Similar to RAID1 but with more disks.</li>
</ul>
<h2 class="mume-header" id="memory-hierarchy">Memory Hierarchy</h2>
<p><strong>Tiering</strong> is a technology that categorizes data to choose different type of storage media to reduce the total storage cost. Tiered storage policies place the <strong>most frequently accessed data on the highest performing storage</strong>. Rarely accessed data goes on low-performance, cheaper storage.</p>
<p align="center">
<img src="./assets/memory-tiering.png" width="600">
</p>
<p><strong>Caches</strong>:</p>
<ul>
<li>CPU Registries</li>
<li>CPU Cache</li>
</ul>
<p><strong>Memory tiering</strong>:</p>
<ul>
<li>RAM</li>
<li>nvRAM (uses <a href="#nvdimm">nvDIMM</a>)</li>
</ul>
<p><strong>Storage tiering</strong>:</p>
<ul>
<li>SSD Memory</li>
<li>Hard drive</li>
<li>Tape</li>
</ul>
<h3 class="mume-header" id="nvme">NVMe</h3>
<p align="center">
<img src="./assets/3d-xpoint-performance.jpg" width="600">
</p>
<p>It's a protocol on the PCI-express bus and it's totally <strong>controller-less</strong>. From the software side it's simpler in this way to talk with the disk because the driver is directly attached to the PCI, there is no controller and minor latency.</p>
<p>A bus is a component where I can attach different devices. It has a clock and some lanes (16 in PCI, ~15 GBps because each lane is slightly less then 1 GB). <strong>Four drives are enough to exhaust a full PCI v3 bus</strong>. They are also capable of saturating a 100 Gbps link, since a NVMe SSD has a bandwidth of 3.5 GBps (3.5*4 = 14 GBps => almost filled the 15 GBps of the PCI-e).</p>
<p>NVMe has now almost totally replaced SATA, since the latter uses 2 PCIe lines and for that reasons represents the bottleneck considering the actual SSD speed.<br>
Furthermore, NVMe is often uses in the lower memory tier of the RAM: its speed is only one order of magnitude less than RAM, but can have a very big size without any problem. For that reason represent a valid super-fast cache level for the RAM and them started being associated in one single level to implement a big RAM tier, in a totally transparent way for the system.</p>
<p>Since the software latency in disk IOs is 5 microseconds more or less, TCP/IP software introduces also a latency of 70-80 microseconds, the disk is no more a problem. Indeed, the problem is now the network, not only for the latency, but also for the bandwidth: 4 NVMe totally saturates a 100 Gbps network.</p>
<h3 class="mume-header" id="nvdimm">nvDIMM</h3>
<p><a href="https://en.wikipedia.org/wiki/NVDIMM">nvDIMM</a> (non volatile Dual Inline Memory Module) is used to save energy. It allows to change the amount of current given to each line, that is as much as a SSD needs to write.</p>
<p>The memory power consumption is a problem, because it usually consume more current than the CPU; moreover the RAM to persists after a reboot needs to be battery-powered, that is very expensive.<br>
With the advent of SSD and NVMe things changed, since we reach high speed with persistent memory: non-volatile memory does not need power unless the need of performing I/O operations; moreover data does not need to be refreshed periodically to avoid data loss.</p>
<p>nvDIMM allows to put SSDs on the memory BUS as for the RAM instead of the PCIe as for the storage.</p>
<h3 class="mume-header" id="misc">Misc</h3>