Merge pull request #767 from EducationalTestingService/release/5.0.0

Release v5.0.0

.readthedocs.yml

@@ -20,4 +20,4 @@ python:
 build:
   os: "ubuntu-22.04"
   tools:
-    python: "3.8"
+    python: "3.11"

README.rst

@@ -49,7 +49,7 @@ You can install using either ``pip`` or ``conda``. See details `here <https://sk
 Requirements
 ~~~~~~~~~~~~
 
--  Python 3.8, 3.9, 3.10, or 3.11
+-  Python 3.10, 3.11, or 3.12.
 -  `beautifulsoup4 <http://www.crummy.com/software/BeautifulSoup/>`__
 -  `gridmap <https://pypi.org/project/gridmap/>`__ (only required if you plan
    to run things in parallel on a DRMAA-compatible cluster)

conda-recipe/README.md

@@ -6,5 +6,5 @@
 
 3. Upload the package file to anaconda.org using `anaconda upload --user ets <path_to_file>`.
 
-4. Test the package: `conda create -n foobar -c ets -c conda-forge python=3.9 skll`. This should _always_ install the latest package from the ``ets`` conda channel.
+4. Test the package: `conda create -n foobar -c https://conda.anaconda.org/ets -c conda-forge python=3.11 skll`. This should _always_ install the latest package from the ``ets`` conda channel.
    Note that we are specifying the ``ets`` channel first since SKLL is now also in conda-forge but runs a version behind until we do the actual release on GitHub.

conda-recipe/skll/meta.yaml

@@ -1,6 +1,6 @@
 package:
   name: skll
-  version: 4.0.1
+  version: 5.0.0
 
 source:
   path: ../../../skll
@@ -24,21 +24,22 @@ build:
 
 requirements:
   host:
-    - python >=3.8
+    - python >=3.10
     - pip
     - setuptools
   run:
-    - python >=3.8
+    - python >=3.10
     - beautifulsoup4
     - joblib
-    - numpy
+    - numpy<2
     - pandas
     - ruamel.yaml
-    - scikit-learn >=1.3.0,<1.4.0
+    - scikit-learn >=1.4.0,<1.5.0
     - scipy
     - seaborn
     - tabulate
     - typing_extensions
+    - wandb
 
 test:
   # Python imports

doc/conf.py

@@ -33,7 +33,7 @@
 
 # General information about the project.
 project = "SciKit-Learn Laboratory"
-copyright = "2012-2023, Educational Testing Service"
+copyright = "2012-2024, Educational Testing Service"
 
 # The short X.Y version.
 version = __version__
@@ -70,7 +70,7 @@
     # The paper size ('letterpaper' or 'a4paper').
     "papersize": "letterpaper",
     # The font size ('10pt', '11pt' or '12pt').
-    "pointsize": "12pt"
+    "pointsize": "12pt",
     # Additional stuff for the LaTeX preamble.
     #'preamble': '',
 }
@@ -112,4 +112,4 @@
 epub_title = "SciKit-Learn Laboratory"
 epub_author = "Educational Testing Service"
 epub_publisher = "Educational Testing Service"
-epub_copyright = "2012-2023, Educational Testing Service"
+epub_copyright = "2012-2024, Educational Testing Service"

doc/contributing.rst

@@ -78,11 +78,11 @@ There are three main entry points into the SKLL codebase:
    `run_experiment <https://skll.readthedocs.io/en/latest/run_experiment.html#using-run-experiment>`__ script. When you run the command
    ``run_experiment <config_file>``, the following happens (at a high level):
 
-   -  the configuration file is handed off to the `run_configuration() <https://github.com/EducationalTestingService/skll/blob/main/skll/experiments/__init__.py#L609>`__ function in ``experiments.py``.
+   -  the configuration file is handed off to the `run_configuration() <https://github.com/EducationalTestingService/skll/blob/main/skll/experiments/__init__.py#L613>`__ function in ``experiments.py``.
 
    -  a `SKLLConfigParser <https://github.com/EducationalTestingService/skll/blob/main/skll/config/__init__.py#L44>`__ object is instantiated from ``config.py`` that parses all of the relevant fields out of the given configuration file.
 
-   -  the configuration fields are then passed to the `_classify_featureset() <https://github.com/EducationalTestingService/skll/blob/main/skll/experiments/__init__.py#L64>`__ function in ``experiments.py`` which instantiates the learners (using code from ``learner.py``), the featuresets (using code from ``reader.py`` & ``featureset.py``), and runs the experiments, collects the results, and writes them out to disk.
+   -  the configuration fields are then passed to the `_classify_featureset() <https://github.com/EducationalTestingService/skll/blob/main/skll/experiments/__init__.py#L65>`__ function in ``experiments.py`` which instantiates the learners (using code from ``learner.py``), the featuresets (using code from ``reader.py`` & ``featureset.py``), and runs the experiments, collects the results, and writes them out to disk.
 
 2. **SKLL API**. Another way to interact with SKLL is via the SKLL API directly in your Python code rather than using configuration files. For example, you could use the `Learner.from_file() <https://github.com/EducationalTestingService/skll/blob/main/skll/learner/__init__.py#L384>`__ or `VotingLearner.from_file() <https://github.com/EducationalTestingService/skll/blob/main/skll/learner/voting.py#L243>`__ methods to load saved models of those types from disk and make predictions on new data. The documentation for the SKLL API can be found `here <https://skll.readthedocs.io/en/latest/api.html>`__.
 

doc/run_experiment.rst

@@ -595,11 +595,27 @@ fixed_parameters *(Optional)*
 """""""""""""""""""""""""""""
 
 List of dictionaries containing parameters you want to have fixed for each
-learner in :ref:`learners` list. Any empty ones will be ignored
-(and the defaults will be used). If :ref:`grid_search` is ``True``,
+learner in the :ref:`learners` list. Empty dictionaries will be ignored
+and the defaults will be used for these learners. If :ref:`grid_search<grid_search>` is ``True``,
 there is a potential for conflict with specified/default parameter grids
 and fixed parameters.
 
+.. note::
+    Tuples are not supported in the config file, and will lead to parsing errors.
+    Make sure to replace tuples with lists when specifying fixed parameters.
+    As an example, consider the following parameter that's usually defined as a tuple in scikit-learn:
+
+    .. code-block:: python
+
+       {'hidden_layer_sizes': (28, 28)}
+
+    To specify it in `fixed_parameters`, use a list instead:
+
+    .. code-block:: python
+
+       {'hidden_layer_sizes': [28, 28]}
+
+
 The default fixed parameters (beyond those that ``scikit-learn`` sets) are:
 
 AdaBoostClassifier and AdaBoostRegressor
@@ -1190,11 +1206,24 @@ SVR
        {'C': [0.01, 0.1, 1.0, 10.0, 100.0]}
 
 .. note::
-    Note that learners not listed here do not have any default
-    parameter grids in SKLL either because there are no
-    hyper-parameters to tune or decisions about which parameters
-    to tune (and how) depend on the data being used for the
-    experiment and are best left up to the user.
+   1. Learners not listed here do not have any default
+      parameter grids in SKLL either because there are no
+      hyper-parameters to tune or decisions about which parameters
+      to tune (and how) depend on the data being used for the
+      experiment and are best left up to the user.
+   2. Tuples are not supported in the config file, and will lead to parsing errors.
+      Make sure to replace tuples with lists when specifying fixed parameters.
+      As an example, consider the following parameter that's usually defined as a tuple in scikit-learn:
+
+    .. code-block:: python
+
+       {'hidden_layer_sizes': (28, 28)}
+
+    To specify it in `param_grids`, use a list instead:
+
+    .. code-block:: python
+
+       {'hidden_layer_sizes': [28, 28]}
 
 
 .. _pos_label:

doc/tutorial.rst

@@ -34,7 +34,7 @@ this is in a virtual environment. For this tutorial, we will
 use `conda <https://conda.io/projects/conda/en/latest/user-guide/install/index.html>`__
 for creating our virtual environment as follows::
 
-    conda create -n skllenv -c conda-forge -c ets python=3.9 skll
+    conda create -n skllenv -c conda-forge -c ets python=3.11 skll
 
 This will create a new virtual environment named ``skllenv`` with the
 latest release of SKLL which you can then activate by running

pyproject.toml

@@ -4,8 +4,12 @@ line-length = 100
 target-version = ['py38']
 
 [tool.ruff]
-select = ["D", "E", "F", "I"]
-ignore = ["D212"]
+extend-exclude = ["setup.py"]
+lint.select = ["D", "E", "F", "I"]
+lint.ignore = ["D212"]
 line-length = 100
 target-version = "py38"
 fix = true
+
+[mypy]
+exclude = '^setup\.py$'

requirements.txt

@@ -2,7 +2,7 @@ beautifulsoup4
 coverage
 joblib
 nose2
-numpy
+numpy<2
 pandas
 pre-commit
 ruamel.yaml

setup.py

@@ -21,7 +21,7 @@ def requirements():
 
 setup(
     name="skll",
-    version=__version__,  # noqa: F821
+    version=__version__,  # type: ignore # noqa: F821
     description=(
         "SciKit-Learn Laboratory makes it easier to run machine "
         "learning experiments with scikit-learn."
@@ -59,10 +59,9 @@ def requirements():
         "Operating System :: Unix",
         "Operating System :: MacOS",
         "Programming Language :: Python :: 3",
-        "Programming Language :: Python :: 3.8",
-        "Programming Language :: Python :: 3.9",
         "Programming Language :: Python :: 3.10",
         "Programming Language :: Python :: 3.11",
+        "Programming Language :: Python :: 3.12",
     ],
     zip_safe=False,
 )

skll/version.py

@@ -10,5 +10,5 @@
 :organization: ETS
 """
 
-__version__ = "4.0.1"
+__version__ = "5.0.0"
 VERSION = tuple(int(x) for x in __version__.split("."))