-
Notifications
You must be signed in to change notification settings - Fork 19
/
EnDe.man.txt
2865 lines (2436 loc) · 115 KB
/
EnDe.man.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
NAME
EnDe - Encoder, Decoder, Converter, Calculator, TU WAS DU WILLST .. [1]
for various codings used in the wild wide web
SYNOPSIS
Click & Enjoy
QUICK START
Type, click & go.
After installation (see below) or opening the proper URL, EnDe can be
used right away without reading the whole manual here. Most fields and
buttons and so on come with a brief description.
Key in your text in the proper input field, select any of the listed
actions in the selection box on the left, and ready you go.
If you feel that you need to know more, just click on [?] button which
should jump to the proper description in the manual.
Or check the EXAMPLES, HINTS below to get some examples right away.
If you want to perform more sophisticated actions, please read the
BEFORE DECODING section below first.
There is a small FAQ in http:EnDe.man.html?EnDe.FAQ.txt .
EnDe does not make any HTTP requests, anything is done in the browser.
Ooops wrong, the manual pages and loading external files are done with
a HTTP request, though.
INTRODUCTION
What is EnDe useful for, what differs from other similar tools?
The answer to this often asked question is manifold, so let's try to
explain with examples.
US-ASCII EXAMPLE
Using the string 'Euro' for some codings and hashes:
+-
| CRC-32 base64 urlUTF-8 MD5(hex)
| ------------+-----------+-----------+---------------------------------
| 1fb0ba15 RXVybw== Euro 3E823FAC7473E42888932C7761C224FC
+-
Trying these codings and hashes in various tools should always return
the same result. Not very surprising, it's expected to do so.
COMMON EXAMPLE
Now let's replace the 'E' by '€' which then is '€uro'.
If all characters are Unicode (UCS-2) the result is as follows:
+-
| CRC-32 base64 urlUTF-8 MD5(hex)
| ------------+-----------+-----------+---------------------------------
| b4c7b4a7 IKx1cm8= %e2%82%acuro 08c1d37a0b3394da278b77116cc4e615
+-
Trying these codings and hashes in various tools most likely returns
different result depending on the tool. See following examples.
OTHER TOOLS
Here is a small list of tools and their results (as for 2010/2011):
+-
| tool base64 urlUTF-8 MD5(hex)
| ----------+-----------+-------------+---------------------------------
| [99] IKx1cm8= %e2%82%acuro 08c1d37a0b3394da278b77116cc4e615
| [29] 4oKsdXJv %E2%82%ACuro 36f9d3bb76279be66536e2d4e250c7ad
| [31] 916f2296b42011db9c5e780b02bd3d5f
| [32] rHVybw== %E2%82%ACuro
| [33] 4oKsdXJv 36F9D3BB76279BE66536E2D4E250C7AD
| [34] 4oKsdXJv %E2%82%ACuro
| [35] 4oKsdXJv %E2%82%ACuro 36f9d3bb76279be66536e2d4e250c7ad
| [36] 4oKsdXJv %E2%82%ACuro 36f9d3bb76279be66536e2d4e250c7ad
| [37] rHVybw== %u20ACuro
| [38] gHVybw== 203b6da413075e767581e4be9354295b
<!--
| [39] 4oKsdXJv 36f9d3bb76279be66536e2d4e250c7ad
-->
| [40] undefinedHVybw== %u20ACuro 08c1d37a0b3394da278b77116cc4e615
| [41] HVybw== %u20ACuro 08c1d37a0b3394da278b77116cc4e615
| [42] 08c1d37a0b3394da278b77116cc4e615
| [43] 3e823fac7473e42888932c7761c224fc
| [44] 4oKsdXJv %E2%82%AC%75%72%6F 36f9d3bb76279be66536e2d4e250c7ad
| [45] 4oKsdXJv 24bd7d579377f38cafec01ee4c8ab976
| [46] gHVybw==
| [47] 4oKsdXJv %E2%82%ACuro 36f9d3bb76279be66536e2d4e250c7ad
| [49] HVybw== %u20ACuro 08c1d37a0b3394da278b77116cc4e615
| [51] 4oKsdXJv 36F9D3BB76279BE66536E2D4E250C7AD
| [52] rHVybw==
| [62] gHVybw== %80uro 203B6DA413075E767581E4BE9354295B
| [63] rHVybw== %ac%75%72%6f 08c1d37a0b3394da278b77116cc4e615
| [64a] 23Vybw== %E2%82%ACuro 226D11B3E7FFC35CCE5C815378A78323
| [64b] gHVybw== %E2%82%ACuro 203B6DA413075E767581E4BE9354295B
| [64c] P3Vybw== %E2%82%ACuro 59DA12B00E4E700E1C5E4B1157FB595B
| [65a] P3Vyb %E2%82%ACuro 4AB85A2FF8E48AB3978DD495F17C0885
| [65b] P3Vyb %E2%82%ACuro E169C0830A05B16A7FB7ED8BEA3548C1
| [66a] P3Vybw== %E2%82%ACuro 36F9D3BB76279BE66536E2D4E250C7AD
| [66b] 4oKsdXJv %E2%82%ACuro 36F9D3BB76279BE66536E2D4E250C7AD
| [66c] 4oKsdXJv %E2%82%ACuro 203B6DA413075E767581E4BE9354295B
| [69] gHVybw== %80uro 203b6da413075e767581e4be9354295b
| [71] gHVyb3== %80uro 203b6da413075e767581e4be9354295b
| [72] gHVyb3== %80uro 203b6da413075e767581e4be9354295b
| [73] gHVybw== €uro 203b6da413075e767581e4be9354295b
| [74] P3Vybw== %E2%82%ACuro 59DA12B00E4E700E1C5E4B1157FB595B
| [75] 4oKsdXJv %e2%82%acuro
| [76]
| [77] 4oKsdXJv %e2%82%acuro 36F9D3BB76279BE66536E2D4E250C7AD
| [81]
| [88] 4oKsdXJvCg== 42dfe0927d2ea6daced8c118bbb8002b
+-
This are 12 different base64 and 12 different MD5 results for 32 tools.
Differences in URL-UTF-8 encodings not counted.
<!--
[32] was ist realurlenc? %20ac%75%72%6f%3c
[33] http://www.zorc.breitbandkatze.de/crc.html CRC-32 1FB0BA15
[35] http://www.prox42.de/produkt/convert/demo.php CRC-32 3035337571 oder 6393ebb4
[36] http://www.webutils.pl/Ascii85 CRC-32 b4eb9363
[73] http://www.nosec.org/web/files/Encoder.exe CRC-32 5E23A421
-->
The results are different. We'll have even more fun when copying (copy
& paste) from one tool to another.
What is wrong?
There is nothing wrong and it's not a bug in those tools.
Just a different interpretation of the given input data. In particular
the charset of the input data.
Online Browser Tools
[29] (2024) https://toolbox.googleapps.com/apps/encode_decode/
[32] (2010) http://www.businessinfo.co.uk/labs/hackvertor/hackvertor.php
[33] (2010) http://textop.us/
[34] (2010) http://h4k.in/encoding/index.php
[35] (2010) http://www.prox42.de/produkt/convert/demo.php
[36] (2010) http://www.webutils.pl/Ascii85
[37] (2010) http://ostermiller.org/calc/encode.html
[38] (2010) http://www.php-einfach.de/base64_generator.php
[39] (2010) http://gtools.org/tool/base64-encode-decode/
[31] (2012) http://rogeriopvl.com/hashr/api2/hash hashr 1.1
Firefox Browser Add-Ons
[40] (2010) https://addons.mozilla.org/firefox/addon/3899 HackBar (Mac OSX)
[41] (2010) https://addons.mozilla.org/firefox/addon/3899 HackBar (Windows)
[42] (2010) https://addons.mozilla.org/firefox/addon/3208 Fire Encrypter (Mac OSX)
[43] (2010) https://addons.mozilla.org/firefox/addon/3208 Fire Encrypter (Windows)
[48] (2010) https://addons.mozilla.org/firefox/addon/baseconvertor/ BaseCorner Converter (Linux)
[49] (2012) https://addons.mozilla.org/en-US/firefox/addon/cryptofox/ CryptoFox 2.2
[31] (2012) https://addons.mozilla.org/en-US/firefox/addon/hashr/ hashr 1.1
Chrome Browser Extensions
[44] (2010) https://chrome.google.com/webstore/detail/iihoicgdaflndmojghlcnfgjkpkhpgee?hl=en-US JavaScript Toolbox (Linux)
[45] (2010) https://chrome.google.com/extensions/detail/gncnbkghencmkfgeepfaonmegemakcol d3coder (Linux)
[46] (2010) https://chrome.google.com/webstore/detail/ochhcobhdebiaimobmlnjogeggcgafgd?hl=en-US# Advanced Encoder/Decode (Linux)
[47] (2013) https://chrome.google.com/webstore/detail/kignjplbjlocolcfldfhbonmbblpfbjb Hasher (Linux)
Opera Widgets (Mac OSX)
[51] (2010) http://widgets.opera.com/category/web-developer/ FastHash, rHash 0.2
[52] (2010) http://widgets.opera.com/category/web-developer/ Text Encoder
Java Tools
[62] (2010) http:// @stake WebProxy (Linux, Mac OSX, Windows)
[63] (2012) http://portswigger.net burp 1.3, 1.4, 1.5
[64a] (2011) http://www.owasp.org/index.php/Category:OWASP_JBroFuzz 2.x (Mac OSX)
[64b] (2011) http://www.owasp.org/index.php/Category:OWASP_JBroFuzz 2.x (Windows)
[64c] (2010) http://www.owasp.org/index.php/Category:OWASP_JBroFuzz 1.8 (Windows)
[65a] (2012) http://code.google.com/p/andiparos/ 1.0.6 (Linux)
[65b] (2012) http://code.google.com/p/andiparos/ 1.0.6 (Windows)
[66a] (2011) https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project ZAP 1.3.4 (Linux, Windows)
[66b] (2012) https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project ZAP 1.4.0.1 (Linux)
[66c] (2012) https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project ZAP 1.4.0.1 (Windows)
[69] (2010) https://www.owasp.org/index.php/Category:OWASP_WebScarab_Project WebScarab 20100414-0036 (Windows)
Windows Binaries
[71] (2011) https://www.bindshell.net/tools/coder/ Coder
[72] (2012) http://www.acunetix.com/ WVS 8
[73] (2011) http://www.nosec.org/web/files/Encoder.exe NOSEC Encoder
[74] (2010) http://parosproxy.org/ Paros Build-in Tool Encode/Hash
[75] (2010) http://www.fiddler2.com/fiddler/ Fiddler
[76] (2010) http://code.google.com/webslayer/ WebSlayer
[77] (2012) http://ironwasp.org/ IronWASP
Other Tools
[81] (2010) http://www.owasp.org/index.php/Category:OWASP_Webslayer_Project WebSlayer
Unix shell (ash, bash, csh, ...) Intel CPU (little endian)
[88] (2011) echo €uro | md5sum
EnDe
[99] (2011) EnDe
DECODE BASE64
We got various different results for encoding. Now let's try to simply
decode base64 but omit the trailing '=' filling characters (which are
not necessary for base64 decoding correctly):
+-
| tool base64 decoded
| ----------+-----------+-------------
| [99] IKx1cm8 â¬uro
| [29] 4oKsdXJv ¿uro
| [32] rHVybw <error>
| [33] 4oKsdXJv â¬uro
| [34] 4oKsdXJv â¬uro
| [39] 4oKsdXJv â¬ur
| [40] undefinedHVybw
| [41] HVybw
| [47] 4oKsdXJv â¬uro
| [48] IKx1cm8= ¿ro
| [48] IKx1cm8
| [49] HVybw== \À
| [51] 4oKsdXJv
| [52] rHVybw 슬urbw
| [62] gHVybw â¬uror
| [63] rHVybw 슬urbw
| [64a] 23Vybw â¬ur
| [64b] gHVybw â¬ur
| [64c] P3Vybw --nothing--
| [65a] P3Vybw ?ur
| [65b] P3Vybw ?ur
| [66a] P3Vybw --nothing--
| [69] gHVybw Exception! java.lang.ArrayIndexOutOfBoundsException: 6
| [71] gHVyb3 â¬uro
| [72] gHVyb3 --nothing--
| [73] P3Vybw --nothing--
| [74] 4oKsdXJv â¬uro
| [74] 4oKsdXJvMQ Error: Ungültige Länge für einen Base-64-Zeichenarray.
| [75] rHVybw
| [81] IKx1cm8 ¬uro
| [81] rHVybw== ¬uro
+-
Unfortunatelly some of the tools even fail to decode correctly without
removing the trailing '='. This is not mentioned in above table.
Please make your own opinion if it's a bug or feature.
This makes 13 different results for 23 tools.
Finally, this introduction was compiled in 2010 and 2011 where these
problems still exist (however, EnDe development began in 2006).
Update 2024: many tools no longer exist or have been corrected. Please
check yourself like: echo €uro | my-encode | my-decode
BROWSER TOOL - EnDe
While having so much trouble to get the right encoding, EnDe was born
so that (hopefully) reliable coding, hashing etc. exists in browsers.
EnDe does not claim to do it the right way, but it at least tells you
how the input data is used (always Unicode, except told otherwise).
As explained below, EnDe's main purpose is for HTTP/HTML processing so
that we don't have such media discontinuity anymore.
TARGET AUDIENCE
EnDe can be used for various duties, the audience may be:
* developers building secure applications
* penetration testers (see also [6], [7])
* analysing malicious web applications
* anyone interested in codings
DESCRIPTION
EnDe is a library with a collection of functions for various codings,
encodings, decodings and conversions.
The objectives are mainly driven by the requirements for functionality
in HTTP/HTML systems. However, it's not limited to that, tell us your
story ...
IT SHOULD ..
* .. be possible to be used in a straight forward way
* .. have mainly just one click for each functionality
* .. be possible to nest functions for continous use
* .. be usable anywhere at any time
IT IS NOT ..
* .. a replacement of all/most of the tools mentioned at LINKS below
* .. a perfect implementation (hey, you're engaged to fix all existing
bugs you find, add new features, improve usability, etc. :-)
* .. a secure browser application
* .. test for RegEx syntax (lint or alike)
* .. converter for RegEx from or to another syntax (flavour)
FURTHER DESCRIPTION
The description of EnDe's library functions can be found at the API
section below. The browser interface to that API is described at GUI
section below, and you find the QUICK START above for lazy people.
CONVENTIONS, TEXT MARKUP
Following text markups are used herein (this help text):
* [button] button is a button in the GUI
* "some text" is either a label in the GUI or a filename
* 'some text' is a literal text
* multiple lines of code, which is literal text too, look like:
+-
| first line
| second line
| ..
+-
* a local link (same domain): http:other.html
* external link (other domain): http://other.tld/page.html
* link pointing to an anchor in this help text: QUICK START
* a numbered list:
# item 1
# item 2
* a unnumbered list like this line
* [ < ] jump to previous location in the help page (browser view only)
* [ ^ ] jump to top of this help page (browser view only)
SECURITY
EnDe was designed with no security in mind as it works without calling
external URLs. It does not use JavaScript's 'eval()' or 'setTimeout()'
except for:
* nested function execution; in GUI only
* "Beautify -> .toSource()"; in GUI only
* "Beautify -> .JSReg" which uses 'eval()' internal, in GUI only
* [eval] action (button) in Functions window; GUI only
<!-- 03/2011 no longer used
* Dean Edward's Packer [5] for compressed JavaScript files;
EnDe online version only
12/2010 no more .json files
* "create menu" from .json files; GUI OPTIONS section in GUI only
-->
This pertains to the GUI only, while the API does not use 'eval()'.
Use EnDe at your own risk.
INSTALLATION
Installation is quiet simple: copy the files from your http:EnDe.tgz
download to any directory and point your browser to http:index.html .
Ready you go.
INSTALLATION for a WEB SERVER
Copy the files from your download to the directory of your web server.
It's recommended that all files are delivered by the web server using
'Charset-Set: UTF-8', though it should work with other charsets also.
In Apache you either have to change "httpd.conf" and restart apache or
use a ".htaccess" file in your directory as follows:
+-
| < Location "/EnDe/">
| AddDefaultCharset utf-8
| AddType text/css;charset=utf-8 .css
| AddType text/html;charset=utf-8 .html
| AddType text/plain;charset=utf-8 .txt
| AddType text/javascript;charset=utf-8 .js
| < /Location>
+-
(depending on the location of this ".htaccess" the '< Location>' scope
is obviously obvious:)
BROWSER CONFIGURATION
The browser should be configured with 'Charset-Set: UTF-8'. Even it
would work with common settings too, lesser problems are expected.
JAVASCRIPT VERSIONS
All JavaScript code relies on ECMA-262 standard (ECMAScript v3).
RECOMMENDATION
For good security reasons most browsers restrict access to files from
the same domain (Same Origin Policy). If files other than those being
part of the GUI should be read (for example with the "Load Local File"
function), then it's recommended that EnDe is installed locally and is
accessed local as file like: file:///path/to/index.html .
Less problems are expected ... but see KNOWN GUI PROBLEMS also.
Following browsers are known to work flawless:
Webkit, Mozilla 1.7.x, Firefox 2.x, Firefox 3.x .. 58.x
For anything else the BROWSER QUIRKS need to be consulted.
Most other browsers do not allow access using file:/// . In that case
EnDe must be provided by a webserver.
BEFORE DECODING
While all encodings work flawless, decoding data to its previous state
(decode data) needs some pre-conditions. That's why some decodings may
return unexpcted results at first glance.
As decoding needs some care, it is recommended to start with decoded
data first, encode it and then decode it, which should result in its
primitive state (that's how the testing is implemented).
If unknown data should be decoded, and the result contains errors then
it might be usefull to remove some charaters at the beginning and then
start the decoding again. Also try to decode just parts -hint: use the
"selection" switch in GUI OPTIONS.
Often data looks like simple encoded data, but is a mix of encodings,
or partially encoded only, or multiple encoded or ...
In such cases the "Fuzzy decoding" actions may help, or one of the
"User functions" actions, see FUNCTIONS.
I. g. humans are much better skilled in deciding to use which decoding
for which part of the data. "Fuzzy decoding" and "User functions" are
just an attempt to manage some common cases and should not considered
perfect.
GUI
"EnDe.html" is the GUI to the 5 groups of functions (called TOOLS
herein). Each tool provides its own set of functions (called ACTIONS
herein) which accept some parameters. Each tool has its own section in
the GUI.
Global functionalities and parameters for all tools are available in
the GUI OPTIONS and API OPTIONS section preceding the five TOOLS
sections.
All in all, EnDe consist of following sections:
* Trace (optional)
* Test (optional)
* GUI Options
* API Options
* Character
* En- / Decoding
* IP Converter
* Timestamp Converter
* RegEx - Regular Expressions
The tools, which are the last 5 sections, are described below.
Additional to these sections, there exist some inset windows which are
used to configure more general functionality. The inset windows are:
* Functions
* Replace Map
* Browser Quirks
Each will be activated with a button in the GUI OPTIONS section.
The look & feel of the GUI depends on the browser. Unfortunately some
browsers behave strange. The GUI can be tweaked to work around some of
these browser quirks. See BROWSER QUIRKS for more details.
GENERAL DESIGN HINTS
All buttons, checkboxes and select menus will show a brief description
when the mouse is over the element (using the tag's 'title' attribute).
If the description is missing, mainly for 'SELECT' tags, then please
refer to KNOWN PROBLEMS for more details.
Any select menu item calls an action which changes something somewhere.
Same applies to all <input size=14 class=button value="(light-blue) coloured"/> buttons.
Whereas all buttons in standard browser style open a new window and do
not change other fields, usually.
Keep in mind that the menus of type SELECT are configured to call the
assigned action when an item receives a click or mouse-up event.
No additional click is required (click & go).
Due to various problems in the behaviour of standard HTML SELECT menus
(the traditional popup menus in one line and the pulldown arrow right)
such menus may not work as expected. See the KNOWN PROBLEMS for more
details.
For customizing the GUI, see CUSTOMIZE GUI below.
COMMON FUNCTIONS
All tools have some actions in common.
On top in the tool's header line:
[-] button to fold the tool's GUI
[x] button to close the tool's GUI (for inset windows)
[?] button to show the description (what you see here)
On bottom of most tools (mainly below the textarea input field):
[-] button quickly clear textarea input field
[reset] button quickly clear input fields (similar to [-] button)
[clear] menu various actions, see TEXT MENU below
[guess] button
[guess] menu to call a guessing action for the tool
[Window ..] to show the current data in a new window (rendered)
[Scratchpad ..] to append the current data to the Scratchpad window
When using the windows ( [Window ..] , [Scratchpad ..] button) you can
preserve various results for later use. The texts can simply be copy &
pasted from these windows. The [Scratchpad ..] window is writable.
Some tools provide 3 tab-like buttons to switched text representations
in their textarea input fields:
[Text] tab
Shows text as keyed in or pasted or generated. For characters
outside the (integer) range 30 .. 127 the displayed character
depends on the browser's implementation and/or the browser's
configuration.
[Hex] tab
Displays the corresponding hex value for each character.
The left part of the display shows the hex values for each of
the characters. The right part shows the character itself (a
dot '.' for none printable US-ASCII characters).
Right and left part are separated by a '|' (bar).
Hex values are one byte usually. Unicode characters greater to
'0xff' are written as 2 bytes without a space in between. This
allows to destinguish 1-byte characters from 2-byte characters
(1-byte: 0 .. 255, 2-byte: 256 .. 65335).
Hint: This can also be used to convert 2 bytes to one Unicode
character, for example: change '20 ac' to '20ac' to get the
Unicode character '€' , the '€' currency sign.
NOTE: only changes in the left (hex) part can be converted to
the [Text] representation.
See also "Special -> od-x style" decoding.
[parsed] tab
Shows text beautified for better (human) reading.
`parsed' means different things in different tools.
In the En-/Decoding section for example it replaces all '&' by
newlines (\n), and all '=' by tabs. In the Functions window it
tries to ident the function parameters and the nested function
calls. For Regular Expressions it tries to ident on braces.
The representations can be switched from [Text], [Hex] or [parsed] to
to [Hex], [parsed] or [Text] or vice versa.
It's recommended that [Text] is selected when using actions.
As described above most tools also provide the [clear] pulldown menu
with actions for special text manipulations/conversions. These actions
apply to the corresponding (above) textarea input field. Each of these
action will be explained with a help text provided by each menu entry.
For more detaisl see the TEXT MENU below.
NOTE: the results of these actions are not reversible.
MODE
EnDe's tool actions operate in one of two modes: "direct" or "nested",
see [Functions] button in GUI OPTIONS (or Quick Bar).
However, currently only en-/decoding supports these modes.
Direct Mode
=======================================
In direct mode all actions are executed immediately, encoded data will
be stored in the Decoding textarea and decoded data will be stored in
the Encoding textarea. This is the default.
Nested Mode (Functions inset window)
=======================================
The FUNCTIONS inset window will be shown and all actions clicked in
Encoding or Decoding are stored as a JavaScript function call in the
window's textarea box. More functions can be found in the select menus
at bottom of the window. Additonally any valid JavaScript can be added
in the texarea field as needed.
This code will then be execuded on demand, just like any other action,
by the buttons found in "show results in ..".
It's called "nested" because each result of an action is used as input
to the next action, this way actions can be chained together.
The generated JavaScript code will simply be stuffed to 'eval()'.
Any valid JavaScript can be added to the generated code as needed. See
EXAMPLES, HINTS below also.
For a detailed description of the Functions inset window see FUNCTIONS
below.
ACTIONS
Actions in the context of EnDe are functions for encoding, decoding or
some kind of conversion of any data (most commonly character strings).
Available actions are mainly the select menu or buttons to the left of
the input fields. For a description see the help text provided by each
seletct (option) entry or button.
Most actions are mapped to a unique function of the API.
Though, some actions are implemented in the GUI only.
TOOLS
All tools have a set of actions, mainly grouped in the select menu to
the left. This menu comes with a description itself for each entry, so
no further description for the actions will be found here.
Most tools also have some special text manipulation functions (mainly
at the bottom of each tool). The detailed description for them can be
found at COMMON FUNCTIONS above and TEXT MENU below.
All tools - their actions in particular - use settings and parameters
from API OPTIONS (see description below).
NOTE: (currently 2011) only the En- / Decoder use these settigs.
The tools are described below with the DETAILS ** headers.
DETAILS Character
Simple conversion of a single character into binary, octal, decimal,
hex, UTF and Unicode representation.
It also contains the [Special Unicode] menu, see UNICODE MENU below
for details.
The [Special Characters] button will show all characters as available
in the menu above, in a new window.
DETAILS En-/Decoding
Encoded texts will be shown in the Decoding textarea and vice versa.
There is no detailed description of each action, the help text should
be self-explanating.
Special Options
=======================================
There're following options, which manipulate the given data in the en-
or decoding textarea before they are passed to the selected action:
"0x00"
Add null byte (NUL) to end of text.
"0x0a"
Add newline (NL) character to end of text.
"0x0d"
Add carriage return (CR) character to end of text.
"0x0a"
Add newline (NL) character to end of text.
"0x1a"
Add end-of-text (SUB) character to end of text.
The "en-/decode" checkbox
"URI/URL"
If checked, text will be en-/decoded before or after selected
action (see action menu) is performed. Text from decoding area
is URL-decoded before, text from encoding area is URL-encoded
after action is performed.
NOTE: These characters are added to the text only before processing.
They are not added to the textarea field.
For practical usage of these options, see EXAMPLES, HINTS below.
Additonally there are the "Sep." radio buttons:
"&"
Use '&' as separator characters in text (Query String style)
":"
Use ':' as separator characters in text (APEX style)
"|"
Use '|' as separator characters in text (GWT style)
"\n"
Use '\n' as separator characters in text (DWR style)
They are used when clicking [Text], [Hex] or [parsed] button.
NOTE: When checked, these buttons provide a simple split of the text
on the selected character only. More sophisticated `parsing' will be
provided in the inset window "User functions", see FUNCTIONS below.
The "UTF8" checkbox
"on"
If checked, decoded data will be UTF8-decoded
(01/2022: used for decoding Viewstates only)
Common Menu
=======================================
Following are common things and those which differ for some actions of
the en-/decoding tool.
For a description of the "Prefix", "Suffix", "Delimiter", "uppercase",
"size", "Key", etc. see API OPTIONS below.
The en-/decoding actions are grouped as follows:
"URI/URL"
All these actions do not use "Prefix", "Suffix" or "Delimiter"
but "uppercase" and "size".
Exception is "Punycode" and "IDN" see below.
NOTE: the "fullwidth (UTF-8)" encoding generates 3-byte UTF-8
characters which most (all?) browsers cannot display. This is
not a bug (in the browser or EnDe) but a general limitation as
JavaScript only supports Unicode (UTF-16).
"HTML-Entity"
All these actions do not use "Prefix", "Suffix" or "Delimiter"
but "uppercase" and "size".
"Unicode/UTF"
All these actions do not use "Prefix", "Suffix" or "Delimiter"
but "uppercase" and "size".
"Base-N"
Various base-N encodings (namely base-64).
All these actions do not use "Prefix", "Suffix" and "Key" but
"wrap" and "mode". Where "wrap" will inserts a newline after
specified number of characters. "mode" (decoding only) behaves
behaves as follows:
* strict - allows only valid characters and requires padding
* lazy - allows only valid characters with optional padding
* verbose - ignores invalid characters (only valid characters
will be decoded), padding is optional. This can be
used for `impedance mismatch' decodings.
<!-- empty line required to keep format; bug in EnDe.man.html -->
"Coding"
Some special codings, see LIMITATIONS below also.
This group also contains "Caesar", "Rot13" and "RotN" which
are encryptions by definition. But it is common practice that
they are treated as codings as the key is well known or simple
to detect.
"Straight"
Some of these codings use the "size" value.
"Numbers"
Simple number conversions.
NOTE: huge numbers may be displayed using exponents.
"Characters"
Does not use any of the options.
"En-/Decryption"
All these function use the value from "Key" as cipher.
The "DES" functions switch automatically between `normal' DES
and tripple DES depending on the length of the given cipher.
If the length is more than 8 characters, tripple DES is used.
The "DES" functions also use the value of "Prefix" as IV.
"Hash/Checksum"
These actions exist in "Encoding" only, obviously.
"RIPEMD-160" can also be found here (even it's not a real hash
function), it uses the value from "Key" as cipher and requires
a value in "Iteration".
For more details see Hashing (MD4, MD5, etc.) below
For CRC functions see description below.
"Escape Characters"
These en-, decodings don't use any API OPTIONS.
"JavaScript built-in"
These actions simply call the browser's built-in functions, so
the results may be different depending on the browser.
NOTE: some functions are not implemented by all browsers.
"Special"
These en-, decodings don't use any API OPTIONS.
"Viewstate" try to parse/fold ASP.NET 1.0 or 2.0 viewstate
For Viewstate the API OPTION "mode" (decoding only) behaves
as follows:
- strict - all values are XML-escaped using:
'EnDe.EN.dispatch("escXML", ...)'
- lazy - all values are printed as is
"xdump style" decode space separated hex values.
Allowed formats:
left side with hex values separated by delimiter right
side any character:
'de,ad,beef | string'
left side with hex values separated by spaces, right
and left side are separated by '|':
'de ad beef | string'
or right side may be missing, but not the separators:
'de ad beef |'
Separator is any character which is not a hex (a-f0-9)
character. Example:
'de ad beef string'
'de ad beef $'
all 4 lines decode the hex values 'de' 'ad' and 'beef'
"od -x style" decode space separated 2- or 4-byte hex values.
Allowed formats same as for "xdump style", but:
The first field and all following spaces are ignored.
Example: prefix=x
'0000 de ad | string'
'00100 beef | string'
decodes the hex values 'de' 'ad' and 'beef'.
"Fuzzy decoding"
These actions exist in "Decoding" only.
For details see Fuzzy Decoding below.
"Symbols"
Some function to convert to `symbols'. The symbols are written
as ASCII-art, hence most of these conversion functions exist
as encoding only. For some fucntions see the [esoteric] button
in GUI OPTIONS below.
See [User API Options] menu also for symbol representations.
"Beautify"
Just some function to make the URL's search part or JavaScript
code human readable.
Following actions print the result in their own window and not
in the En-/Decoding textarea:
"JSDecode" to make obfuscated JavaScript human readable.
"JSFormat" to make compact JSON or JavaScript human readable.
While "JsDecode" is more usefull for JavaScript, "JSFormat" is
better for JSON data (in the author's opinion:).
"JSReg" parse given source with JSReg sandbox.
"show as image" try to display data as image.
[guess] menu
This button contains a menu consisting of above menu groups.
Selecting such a submenu allows guessing for just that group
of actions.
Punycode, IDNA Punycode
=======================================
Punycode and IDNA Punycode is usefull for hostnames (FQDN) only. Each
part of the FQDN needs to be en-/decoded separately. That is what the
standard "Punicode" and "(xn--)IDN" actions do.
Some systems provide the `libidn' with the corresponding `idn' program
which adds the '-' delimiter as trailing character if no character was
encoded. To support this behaviour the "Punicode (libidn compatible)"
[User API Options] was added, which passes "Suffix" for this delimiter
to the actions.
NOTE that some versions of `libidn' are buggy; they may bail out with
an error or even crash when decoding some strings. Also note that some
versions always encode the complete string instead of splitting at the
'.' (dot). Such bugs are not implemented in EnDe ;-)
CRC
=======================================
There are various implementations of different CRC - Cyclic Redundancy
Codes/Checks - algorithms to be found in the wild. It is difficult to
explain all the variants and some of them are even confusing.
For a detailed description see [8].
The basic idea of these algorithms are:
* there is an initial vector
* compute the checksum of the given data using ploynomial arithmetic
* an output mask may be added to the result.
Furthermore all the used data may undergo some transformations called
`reflection' [*] like:
* reflect the data per byte
* reflect the data as a whole
* reflect each byte of the data and then byte-vice as a whole.
These reflections can be performed on:
* the inital vector
* the ploynom (which is in fact the same as using different values)
* the given data
* the mask for the result
* the result itself (before finally XOR'ed).
The algorithms can be done by coding loops for processing the data, or
by using prebuild transformation tables instead of processing, which
then results in better performance.
And as this is not enough confusion, some people mix the inital vector
and the ploynomial which results in 2 modes of the algorithm called:
* direct
* non-direct
[*] reflect - swapping all of its bits around the central point
EnDe uses the coded algorithm and table driven algorithms depending on
the selected CRC type. Only a few types are supported directly.
The full support for algorithmic and table driven calculations is only
available in the API , while the GUI is limited to the most common
CRC functions used in pratice.
The API can be used directly for example in "nested" mode by writing
the corresponding function calls there (details see "EnDeFunc.txt").
However, there is a workaround in the GUI to supports some parameters:
"Prefix"
the value for the initial vector (hex digits prefixed '0x')
"Suffix"
the value for the output mask (hex digits prefixed '0x')
"Key/Cipher"
the value for the polinomial (hex digits prefixed '0x')
or a constant for using predifined tables:
- 'ARCtab'
- 'MODtab'
- 'PPPtab'
<!-- NOT YET IMPLEMENTED
- 'C32tab'
Delimiter
following fixed key can be used:
- 'REFIN'
- 'REFOUT'
- 'REVERSE/REFLECT'
-->
NOTE: the "values are hex" checkbox is not observed.
Some common used combinations are defined in the [User API Options]
menu and can be used with the "CRC (user)" action. They are useful as
example if special CRC types should be crafted.
The avaialable encoding actions "CRC-8", "CRC-16", "CRC-16(CCITT)" and
"CRC-32" do not use these parameters (as they are already hardcoded).
NOTE also that these actions return the result hex-encoded without any
prefix, suffix or delimiter. If the raw (integer) values are required,
the "nested" mode must be used.
The GUI to the CRC functions may (needs to:) be improved in future.
Hashing (MD4, MD5, etc.)
=======================================
Most of the hashing functions are awailable to return their result in
"raw", "hex" or "base64" format.
The GUI to these functions may be improved in future so that there is
only one action and the mode ("raw", "hex" or "base64") is selected by
an option elsewhere.
Symbols (ASCII 3x5)
=======================================
The en-/decoding functions "ASCII 3x5 digit" encode digits to a 3x5
matrix with characters, or decode such a matrix back to the digit.
Thesse functions use the "Prefix", "Suffix" parameters to add leading
or trailing spaces to each matrix. Therefor these parameters must only
contain spaces. Otherwise one space will be used for each.
If "uppercase" is set true, only upper characters and digits will be
used. This is the original mode of the `MathGuard' implementation.
Otherwise upper case, lower case characters and digits will be used.
While decoding, empty columns (means those only containing spaces) are
treated as matrix separator and therefore ignored.
Fuzzy Decoding
=======================================
All en-/decoding functions process each charcater or string pattern in
the given string. If there are different encodings used in the string,
they cannot be decoded proper as most decoding functions may return an
error if a pattern does not match the selected coding.
The fuzzy decoding functions try to circumvent these oddities.
The idea behind fuzzy decoding is that most codings are restricted to
only some parts of the string. Most languages use single quotes ' and/
or double quotes '"' to enclose literal strings.
Currently (04/2011) all fuzzy functions perform decoding inside quoted
strings only (where EnDe supports braces as `quotes' too).
Inside strings fuzzy decoding tries to detect the selected coding and
then passes the matched string to the proper decoding function and add
the result of the function, if any, to the decoded string.
To avoid improper matches, fuzzy decoding uses several decoding types.
For example:
* \-escaped for octal codings
* \x-escaped for hex codings
* %-escaped for URL codings
* HTML Entity for HTML codings
Each function tries to match the longest possible pattern first, then
tries to match the next pattern.
The functions are restricted to either decode inside single quotes or
inside double quotes. If both is required, both functions need to be
called in two separate steps.
Fuzzy decoding can also be used with the prefix, suffix and delimiter
settings from API OPTIONS. In this mode decoding is done for strings
enclosed in prefix and suffix/delimiter.
NOTE that fuzzy decoding hex strings decodes 2 characters, while fuzzy
decoding decimal or octal strings assumes 3 characters (hence leading
zeros may be necessary).
Some useful combinations of fuzzy decoding functions can be found as
user functions in "EnDeUser.xml" which are available in the Functions
inset window's "User functions", see FUNCTIONS .
DETAILS IP Converter
Should be self-explanating. The menu on left uses the mouse-over event
to highlight the input field used as source for the selected action.
Each IP conversion fills all fields. <!-- except "IP in reverse order" -->
DETAILS Timestamp Converter
Each Timestamp conversion also fills all fields. <!-- except "Ref. Time" and "Offset" -->
Timestamp has it's own description see http:EnDe.man.html?EnDe.TS.html
DETAILS RegEx
Explain and pretty print regular expression.
Regular expressions (aka RegEx) are used in many tools and programming
languages. Unfortunately there exist dozens of different languages,
flavours and syntax notations to express the RegEx. This tool tries to
explain the given RegEx in human understandable words and phrases.
Therefore it uses the common known phrases (mainly those used in POSIX
and/or PCRE descriptions).
Each action from menu opens a window with such an explanation for the
RegEx (found in RegEx textarea). This window also displays the result
of that RegEx matched against the text found in Text textarea.
RegEx has it's own description see http:EnDe.man.html?EnDeRE.man.txt .
DETAILS Test
Use the [Test] button or the '?test' option to display this section.
This section provides some test functionalities for EnDe's functions.
Following buttons are available:
[run tests from file]
Loads file with test patterns and performs all tests specified
in the file, then displays the test results.
The default file with the test patterns is "EnDeTest.txt". Any
other file specified in "Load File" [Menu] can be used. Just
the filename needs to be specified and no further button used
there (see LOADING FILES for details).
[run tests from input]
Instead of reading the test patterns from a file (as described
above) they will be read from the Encoding textarea.
Hint: any file can be loaded to the Encoding textarea with the