README.md

Deprecation notice

This part of the repository has in large part been superceeded by various training repositories dedicated to specific Python-related topics.

• Python for programmers
  • repository
• Scientific Python
  • website
  • repository
• Python for data science
  • website
  • repository
• Python for machine learning
  • website
  • repository
• Python for HPC
  • website
  • repository
• Python for systems programming
  • website
  • repository
• Python software engineering
  • website
  • repository

Python

Python is an excellent programming language for data processing, prototyping and writing glue code for C/C++/Fortran libraries.

This directory contains code samples that illustrate particular features and programming techniques, or how to use Python iibraries. Many of these samples have been used for Python tutorials.

What is it?

1. Autocorrelation: illustration of autocorrelation in a Markov process, implements a Monte Carlo algorithm, and allows to show evolution over time, autocorrelation, and to compare with target distribution.
2. Biopython: some Biopython examples.
3. biopython.pptx: Microsoft Powerpoint presentation on using BioPython for bioinformatics related tasks.
4. Birdsong: implements a FFT of WAV files, and shows the original signal (amplitude) and the frequency spectrum using Matplotlib. Another script illustrates elementary signal processing using a highpass filter.
5. Bokeh: illustrations of how to use the Bokeh plotting library for, e.g., interactive graphs.
6. CodeCoverage: illustration f how to use a code coverage tool.
7. CodeEvaluation: illustrates how to evaluate a string containing Python code at runtime.
8. CodeTesting: illustration of how to use the unittest framework for setting up and executing unit tests
9. CommandLineArgs: illustration of how to use the argparse and the click module to handle command line arguments.
10. ConfigParser: illustrates how to use the ConfigParser module to handle configuration files.
11. ContextManager: illustrates the concept of a context manager, a mechanism to deal with resource setup/teardown.
12. Coroutines: illustrates coroutines, functions that preserve state between invocations.
13. Cython: Powerpoint presentation on Cython, as well as example code.
14. DataFormats: illustrates how to deal with data formats such as CSV files, binary data and XML.
15. DataStructures: illustration of some data types defined in Python's collections standard library module and other standard library packages.
16. DbAccess: illustrates how to insert data into a relational database and query it, both with Python's sqlite3 module, and SQLAlchemy.
17. Debugging: some code that is useful for debugging demos.
18. Decorators: illustrates how to create decorators to wrap functions.
19. Descriptors: illustrates the descriptor concept in Python
20. DynamicModuleLoad: illustrates how to load modules at runtime by name.
21. Exercises: example solutions to exercises to be found in the Powerpoint presentation; this is part of the "Python as a data processing language" training session.
22. FiniteStateParser: illustrates how to parse a file that is structured as a regular language. A pyparsing example is also given.
23. Flask: some illustrations of using the Flask web services framework.
24. Functional: some illustrations of functional programming style.
25. Fundamentals: code samples that where used to extract fragments in the part of the presentation on core Python features.
26. Gis: code samples for dealing with GIS data.
27. Interfacing_C_C++_Fortran: illustrates how to use Fortran code from Python using f2py, C code by using ctype, and C/C++ code using SWIG.
28. HoloViews: data visualization library examples, especially in the context of exploration in notebooks.
29. Hydra: Facebook Hydra application framework illustration.
30. ImageProcessing: some Scikit-Image sample code for image processing, as well as OpenCV for video analysis.
31. Interfacing_C_C++_Fortran: illustrations of how to interface with C, C++ and Fortran code using ctypes, SWIG or f2py.
32. Introspection: sample code of how to implement introspection as mix-in class.
33. Ising: Implementation of the 2D Ising system in pure Python, as well as in C++ using SWIG to wrap the implementation. Mostly intended for visualization purposes.
34. Joblib: illustrates the use of this library for easy task parallelism
35. Jupyter: A few notebooks to illustrate the Jupyter notebook GUI.
36. Keras: illustration of using the Keras framework for machine learning tasks.
37. Logging: illustration of Python's logging facilities.
38. Matlab: illustrations of how to call MATLAB functions from Python.
39. Matplotlib: a few illustrations of how to use the pyplot module.
40. Matrices: a few timings of matrix-matrix multiplications using LoL in Python, the numpy.dot function, a Fortran matmul and DGEMM, and a straightforward C implementation.
41. Mixin: illustrates the object-oriented programming technique of mix-ins.
42. Mpi4py: illustration of writing distributed applicationns using MPI
43. Multiprocessing: illustrates concurrent computations using the multiprocessing library.
44. NetworkX: illustration of the graph representation and algorithms library.
45. Nltk: some initial Natural Language ToolKit experiments.
46. Numpy: a few illustratios of using numpy, scipy, and matplotlib (linear regression, optimization, solving ODEs, FFT).
47. OperatorFunctools: some illustrations of the functional programming options offered by opeator and functools packages.
48. OsFileSystem: illustrations of interacting with the operating system and the file system.
49. Pandas: R dataframe-like data structures for Python, contains a Jupyter notebook to illustrate using pandas interactively.
50. Paramiko: a few examples of using the Paramiko library for SSH to remote hosts.
51. Pitfalls: illusttrations of pitfalls when coding Python due to non-trivial semantics.
52. PhraseIndexer: parses a text file to find the line number on which a given eet of phrases occurs.
53. PyCuda: Jupyter notebooks illustrating PyCUDA.
54. Profiling: a few examples of how to use profilers for Python.
55. python_development_practices.pptx: Powerpoint presentation on some development good practices.
56. python_hpc.pptx: Powerpoint presentation on how to use Python for high performance computing, contains sections on Cython, using C/C++/Fortran libraries from Python, shared memory programming, MPI, and PySpark.
57. python_intro.pptx: Powerpoint presentation on Python as a data processing language, exercises for the corresponding training session can be found in the Exercsises directory.
58. Sched: scheduled execution of funcitons in Python.
59. ScikitLearn: examples using the scikit-learn machine learning library.
60. Regexes: some material related to using regular expressions in Python.
61. Seaborn: illustrations of the Seaborn plotting library.
62. SentenceCounter: scripts to (naively) count sentences in natural language text, both serial and parallel implementations.
63. style.md: some style tips based on code reviews and observations in the field
64. SpaCy: illustrations of using the spaCy natural language processing library.
65. Subprocess: illustrates executing a shell command from a Python script using the subprocess module.
66. Typing: Python type annotations can help to make code considerably more robust.
67. Unittest: illustration of how to use Python's standard library unit testing framework.
68. Vtk: examples of using the KitWare VTK library (No support for Python 3 yet).
69. WebScraping: illustration of web scraping using Beautiful Soup.
70. WxPython: some illustrations of GUI developlement using the Python bindings for Wx (No support for Python 3 as yet).
71. Xarray: illustration of using xarray to represent numerical data.
72. XmlGenerator: code to generate a random XML documents.

How to

• Instructions for installing conda and hence Python package managenment on the VSC cluster are available.
• Instructions on how to use a Jupyter notebook server that runs on a compute node on the VSC cluster.