Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update develop #109

Merged
merged 199 commits into from
Nov 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
199 commits
Select commit Hold shift + click to select a range
2d39b14
Merge pull request #44 from mjanez/develop
mjanez Apr 2, 2024
743941f
Merge pull request #45 from mjanez/develop
mjanez Apr 3, 2024
05841ef
Merge pull request #46 from mjanez/develop
mjanez Apr 4, 2024
7143d9a
Merge pull request #48 from mjanez/develop
mjanez Apr 9, 2024
bdd7146
Merge pull request #49 from mjanez/develop
mjanez Apr 10, 2024
aedab05
Merge pull request #51 from mjanez/develop
mjanez Apr 11, 2024
d41aa16
Merge pull request #52 from mjanez/develop
mjanez Apr 27, 2024
36608f3
Merge pull request #53 from mjanez/develop
mjanez May 16, 2024
6a13509
Merge pull request #54 from mjanez/develop
mjanez May 16, 2024
41b5afb
Merge pull request #55 from mjanez/develop
mjanez May 17, 2024
627a25e
Merge pull request #56 from mjanez/develop
mjanez May 17, 2024
861a05a
Merge pull request #57 from mjanez/develop
mjanez May 19, 2024
aac5a64
Merge pull request #58 from mjanez/develop
mjanez May 19, 2024
5026846
Merge pull request #59 from mjanez/develop
mjanez May 19, 2024
c880770
Merge pull request #60 from mjanez/develop
mjanez May 21, 2024
447bcc2
Merge pull request #61 from mjanez/develop
mjanez May 21, 2024
b541759
Merge pull request #62 from mjanez/develop
mjanez May 27, 2024
1681777
Merge pull request #63 from mjanez/develop
mjanez May 27, 2024
feb7b5f
Merge pull request #65 from mjanez/develop
mjanez May 28, 2024
1f112d1
Merge pull request #66 from mjanez/develop
mjanez May 28, 2024
3d227b0
Merge pull request #67 from mjanez/develop
mjanez May 28, 2024
6794880
Merge pull request #68 from mjanez/develop
mjanez Jun 7, 2024
3381418
Merge pull request #69 from mjanez/develop
mjanez Jun 7, 2024
19a3478
Merge pull request #70 from mjanez/develop
mjanez Jun 7, 2024
132c50a
Merge pull request #71 from mjanez/develop
mjanez Jun 7, 2024
b56146a
Merge pull request #72 from mjanez/develop
mjanez Jun 7, 2024
6bf6f59
Merge pull request #73 from mjanez/develop
mjanez Jun 13, 2024
e4869d4
Merge pull request #74 from mjanez/develop
mjanez Jun 15, 2024
7a11b96
Merge pull request #75 from mjanez/develop
mjanez Jun 15, 2024
1aa2bf2
Merge pull request #76 from mjanez/develop
mjanez Jun 17, 2024
6413e6f
Merge pull request #77 from mjanez/develop
mjanez Jun 17, 2024
4531fa4
Merge pull request #78 from mjanez/develop
mjanez Jun 17, 2024
fd91797
Merge pull request #79 from mjanez/develop
mjanez Jun 17, 2024
ff061e5
Merge pull request #81 from mjanez/develop
mjanez Jun 27, 2024
a67b050
Merge pull request #82 from mjanez/develop
mjanez Jul 2, 2024
e777613
Merge pull request #83 from mjanez/develop
mjanez Jul 9, 2024
28163ed
Merge pull request #84 from mjanez/develop
mjanez Jul 10, 2024
bab7d58
Add form tabs to schemas and templates
mjanez Jul 25, 2024
85b79e4
Merge pull request #85 from mjanez/develop
mjanez Jul 30, 2024
9c9a45e
Merge pull request #89 from mjanez/develop
mjanez Jul 30, 2024
28c3d3d
Merge pull request #91 from mjanez/develop
mjanez Jul 31, 2024
ea133a3
Add licenses.json
mjanez Aug 1, 2024
b363a37
Merge pull request #94 from mjanez/feature/dcat-ap-schemas
mjanez Aug 1, 2024
49c4d24
Fix translated_fields generation
mjanez Aug 1, 2024
2128d6c
Fix file_size in resource metadata info
mjanez Aug 1, 2024
57736d4
Merge branch 'feature/form-tabs' into main
mjanez Aug 2, 2024
2da1c1f
Add form tabs to dataset creation/editing datasets/resources
mjanez Aug 4, 2024
9bb1e8d
Merge branch 'main' into feature/dcat-ap-schemas
mjanez Aug 4, 2024
0e63426
Improve schema geodcat_ap_es to include form_tabs
mjanez Aug 4, 2024
f980580
form_tabs_allowed is not optional
mjanez Aug 4, 2024
34aedfb
Updated schema names and improved validators
mjanez Aug 14, 2024
d949571
Add ckanext-dcat custom profiles
mjanez Aug 22, 2024
8f0e6f3
Add SHACL tests for custom profiles
mjanez Aug 22, 2024
bb9f4fd
Fix F821 undefined name
mjanez Aug 22, 2024
b5dc788
Update test.ini
mjanez Aug 22, 2024
e1e488c
Add deprecated decorator to mark functions as deprecated
mjanez Aug 22, 2024
f3f655f
Refactor dataset graph
mjanez Aug 22, 2024
617d3ee
Update ckan DCAT-AP JSON samples
mjanez Aug 22, 2024
d062580
Refactor access rights handling in EuDCATAPProfile
mjanez Aug 22, 2024
2184aad
Add reference and representation_type
mjanez Aug 22, 2024
5091c67
Fix test error, add harvest to plugins
mjanez Aug 22, 2024
3ff0417
Fix test_shacl
mjanez Aug 22, 2024
af0710a
FIx test_shacl & update example ckan datasets
mjanez Aug 22, 2024
a29d367
Refactor test command to include logging level in pytest for ckanext.…
mjanez Aug 22, 2024
5d9a305
Refactor test command to use correct utils in pytest for ckanext.sche…
mjanez Aug 22, 2024
ba289eb
Refactor test command to include fluent plugin in tests
mjanez Aug 22, 2024
ce772ba
Refactor access rights handling in EU DCAT-AP profile
mjanez Aug 22, 2024
e635d71
Todo legacy datasets
mjanez Aug 22, 2024
2b4ddad
FIx shacl file path to include version subdir
mjanez Aug 22, 2024
13d12eb
Refactor GitHub Actions workflow to only trigger on pull requests
mjanez Aug 22, 2024
3eb49e7
Include harvest plugin in tests
mjanez Aug 22, 2024
29f80bd
Improve error reporting in SHACL validation tests using pytest.fail
mjanez Aug 22, 2024
f1841e8
Refactor test_shacl.py to simplify error reporting
mjanez Aug 22, 2024
0b2d0cc
Fix rights bug
mjanez Aug 22, 2024
75a45bd
Fix eu_dcat_ap profile
mjanez Aug 23, 2024
7a51f3d
Improve es_dcat (NTI-RISP) Profile
mjanez Aug 23, 2024
dcd144e
FIx templates with new profiles
mjanez Aug 23, 2024
2386bc7
Refactor field mapping in SchemingDCATHarvester
mjanez Aug 23, 2024
ff4de2c
Add codelists for DCAT profiles
mjanez Aug 23, 2024
5c183ac
Add conftest.py for testing with clean database
mjanez Aug 27, 2024
9b16a8f
Add docker-compose tests
mjanez Aug 27, 2024
4b5c4b7
Merge pull request #100 from mjanez/feature/docker-tests
mjanez Aug 27, 2024
c294495
Fix search and index to ensure proper functionality
mjanez Sep 10, 2024
7fb647a
Improve profiles and add DCAT-AP/GeoDCAT-AP 3
mjanez Sep 10, 2024
0d6559b
Improve schemas to align names with ckanext-dcat
mjanez Sep 16, 2024
8cb65ff
Improve profiles and add DCAT-AP 3
mjanez Sep 16, 2024
4810b07
Improve general UI, Home
mjanez Sep 16, 2024
ef76bb3
Update package/resource forms to Tabs
mjanez Sep 16, 2024
4dca2e4
Improve i18n and icons
mjanez Sep 16, 2024
c7aabef
Fix and improve templates for CKAN 2.10
mjanez Sep 16, 2024
f358203
Fix fa icons for license/social templates
mjanez Sep 16, 2024
7c303d9
Fix before_update
mjanez Sep 16, 2024
fdf55e8
Improve package_item.html template
mjanez Sep 16, 2024
a344aec
Improve base admin config template
mjanez Sep 16, 2024
006078e
Update configuration and add stats init to plugin
mjanez Sep 16, 2024
3da105a
Refactor code to remove debug logging statements
mjanez Sep 16, 2024
6954453
Refactor download metadata template and improve metadata profiles
mjanez Sep 16, 2024
0ffa59e
Improve metadata_info and profile badges
mjanez Sep 16, 2024
c4d131b
Refactor new_source_form.html to use Bootstrap 5 data attributes
mjanez Sep 16, 2024
e136909
Add docs and templates for harvesters
mjanez Sep 16, 2024
69e04f3
Add xls_template_config
mjanez Sep 16, 2024
66dc66d
Improve style to improve header language selector dropdown menu
mjanez Sep 16, 2024
1af58e1
Fix tabs slug-preview and dataset-map
mjanez Sep 16, 2024
fb8abf6
Fix Solr indexing by converting dict fields to JSON strings
mjanez Sep 18, 2024
e3bb1ee
Update open data stats: Skip statistics update during DB initializati…
mjanez Sep 18, 2024
0cbb625
Improve form tabs
mjanez Sep 18, 2024
448113e
Fix package snippets
mjanez Sep 18, 2024
4b10208
Fix organization create/edit button bugs
mjanez Sep 18, 2024
3026467
Fix spatial_query snippet bugs
mjanez Sep 18, 2024
2c02458
Fix xloader logs styles for 2.10
mjanez Sep 18, 2024
50945d0
Fix tags containers
mjanez Sep 18, 2024
537504c
Improve datastore/data api and downloads templates
mjanez Sep 18, 2024
767fa65
Update version to 4.0.0
mjanez Sep 19, 2024
bac0862
Fix xloader logs styles for 2.10
mjanez Sep 19, 2024
6dc1e4c
Improve about_themes.html template to conditionally render theme cards
mjanez Sep 19, 2024
f2a0ed8
Improve tests
mjanez Sep 19, 2024
ddc4de6
Removed unnecessary data-bs attributes for custom slug-preview module
mjanez Sep 20, 2024
fcec54b
Fix api button accordions
mjanez Sep 20, 2024
eefdcac
Add highlighting and quote alerts
mjanez Sep 20, 2024
53f9dc1
Improve admin pages
mjanez Sep 20, 2024
8f22343
Include Update Internationalization (i18n) Files guide
mjanez Sep 20, 2024
1e504ca
Update i18n files and templates for internationalization support
mjanez Sep 20, 2024
32300b8
Refactor social account configuration in SchemingDCATPlugin
mjanez Sep 20, 2024
4bfcea4
Update test to use custom ckan-solr image
mjanez Sep 21, 2024
e3d81ff
Update CKAN version to 2.10 in Test dockerfile
mjanez Sep 21, 2024
5cd2bef
Improve README images
mjanez Sep 22, 2024
3a68063
Merge pull request #101 from mjanez/feature/2.10-update
mjanez Sep 23, 2024
d5582e7
Merge pull request #104 from mjanez/feature/dcat-ap-schemas
mjanez Sep 23, 2024
98ccd3c
FIx harvest source form
mjanez Sep 23, 2024
1f2bc9d
Update README image link to use correct path
mjanez Sep 23, 2024
f3873ea
Fix es_dcat PeriodOfTime class if not exists temporal_*
mjanez Sep 25, 2024
c45eaca
Fix current_url assignment in schemingdcat_social.html
mjanez Sep 26, 2024
140c49d
Improve profile badges and responsive
mjanez Sep 26, 2024
eda46f8
Fix request.params to request.args
mjanez Sep 26, 2024
74de272
Improve endpoints templates
mjanez Sep 26, 2024
4901d25
Refactor ckanext-schemingdcat codelists function to handle NoneType v…
mjanez Sep 26, 2024
cd74774
Refactor default field overrides in config.py
mjanez Sep 26, 2024
9d06db1
Improve README.md
mjanez Sep 26, 2024
72e7d6e
Improve footer css
mjanez Sep 26, 2024
04e2fa0
Improve flexbox layout in themes card container
mjanez Sep 26, 2024
c9f76e0
Update dcs to MKDocs files and update workflows
mjanez Oct 4, 2024
aa49986
Add config_declaration CKAN >2.10
mjanez Oct 5, 2024
4b8c741
Add config_declaration CKAN >2.10 and refactor config.py to config/
mjanez Oct 5, 2024
fda278b
Update endpoint codelists.
mjanez Oct 5, 2024
6bf0ba0
Add signals for package updates and create statistics schema interface
mjanez Oct 5, 2024
e019582
Add statistics logic and templates for SchemingDCAT integration
mjanez Oct 5, 2024
5fa65f9
Improve i18n
mjanez Oct 5, 2024
a85808b
Add initial implementation for SchemingDCAT logic and update actions
mjanez Oct 5, 2024
957a2e6
Refactor spatial resolution field labels and help of schemas
mjanez Oct 5, 2024
e4579f4
Refactor plugin imports and update helper function calls in SchemingD…
mjanez Oct 5, 2024
b2ab4f4
Improve helpers
mjanez Oct 5, 2024
618fe81
Improve theme
mjanez Oct 5, 2024
c47d62a
Add percentage and spatial resolution display snippets, and update sp…
mjanez Oct 5, 2024
b8d9f69
Update MANIFEST to include additional file types and add new SVG icon…
mjanez Oct 5, 2024
705b459
Merge branch 'feature/2.10-update' of https://github.com/mjanez/ckane…
mjanez Oct 5, 2024
c5610c2
Enhance SchemingDCAT extension with new SQLite harvester (Postgres, S…
mjanez Oct 7, 2024
9661186
Fix PostgreSQL database harvester description
mjanez Oct 7, 2024
7a2dfd6
Add Bootstrap class to title input in new source form template
mjanez Oct 7, 2024
f54f69f
Refactor dropdown menu in new source form and fix confirm-actions
mjanez Oct 7, 2024
cf3d53a
Update URL input help text in new source form to include SQL Harveste…
mjanez Oct 7, 2024
f43eec2
Fix nested contact/author/maintainer/publisher fields
mjanez Oct 7, 2024
c92fdb3
Fix bug in SQL Harvester URL input help text in new source form
mjanez Oct 7, 2024
773c2fe
Add debug logging for full SQL query generation and remove unnecessar…
mjanez Oct 8, 2024
d05b4dd
Add number_size display snippet
mjanez Oct 9, 2024
2579f47
Implement pagination in get_theme_datasets to efficiently retrieve al…
mjanez Oct 11, 2024
f60645d
Improve HVD category selection snippet to form templates
mjanez Oct 14, 2024
0e22baf
Add dataset citation snippets and improve permanent URL handling
mjanez Oct 16, 2024
fda5a37
Optimice CSS
mjanez Oct 16, 2024
7ecb4fd
Fix package update logging and event emission in SchemingDCATHarvester
mjanez Oct 16, 2024
9388397
Update i18n
mjanez Oct 16, 2024
e74a4f8
Merge pull request #107 from mjanez/feature/bibliographic-info
mjanez Oct 16, 2024
294dcff
Add Luxembourgish language support and remove duplicates in DCAT schemas
mjanez Oct 16, 2024
045f164
Optimize bibliographic type check and improve template logic for data…
mjanez Oct 17, 2024
1918b6a
Refactor the SchemingDCATHarvester to use the configuration defaults …
mjanez Oct 17, 2024
48b95c8
Add BibTeX and RIS download functionality and enhance citation layout
mjanez Oct 22, 2024
9935d21
Enhance dataset citation template to improve resource creation date h…
mjanez Oct 23, 2024
dcb88d3
Add dataset_id_field to SchemingDCATXLSHarvester for identifier resol…
mjanez Oct 23, 2024
86e16a9
Add translations for "Parent" and "None - top level" in English and S…
mjanez Oct 23, 2024
00f5eb1
Rename import for schemingdcat_get_schema_names to improve clarity
mjanez Oct 23, 2024
af8c4cc
Fix organization name retrieval to use display_name
mjanez Oct 23, 2024
8de31b9
Removed old logic.py
mjanez Oct 23, 2024
dae2d83
Update JSON templates to standardize field names and add applicable l…
mjanez Oct 23, 2024
617db41
Refactor get_actions methods to simplify return statements and remove…
mjanez Oct 23, 2024
47aafe3
Update README to reflect compatibility with CKAN version 2.9
mjanez Oct 23, 2024
5af10af
Extend private field with publisher logic
mjanez Oct 24, 2024
b371495
Fix dataset privacy settings logic and improve authorization messages
mjanez Oct 24, 2024
84a1ab5
Using fixed schemingdcat_member_list instead of core member_list
mjanez Oct 24, 2024
b1a1b32
Comment debug logging from user organization membership checks and au…
mjanez Oct 24, 2024
1974d33
Update README and enhance visibility logic for publisher roles; refac…
mjanez Oct 25, 2024
7b86e3c
Fix validators for dataset privacy settings (private)
mjanez Oct 29, 2024
68e810e
Comment out debug logging for content dictionary and update warning l…
mjanez Oct 29, 2024
81c8e08
Update access rights preset in multiple schemas to use 'access_rights'
mjanez Oct 29, 2024
d7a1907
Update license form snippet to using default value CC-BY-4.0 to all s…
mjanez Oct 29, 2024
1705773
Fix dataset_id_field assignment in SchemingDCATXLSHarvester configura…
mjanez Nov 4, 2024
ef1f26c
Fix access_rights, maintainer and author field bugs
mjanez Nov 4, 2024
7c9e91f
Add XLS_HARVESTER_FIELDS_NOT_LIST to exclude fields that are not list…
mjanez Nov 4, 2024
4e31554
Update accrualPeriodicity choices to include 'Provisional data' as de…
mjanez Nov 4, 2024
6504fb6
Update CHANGELOG for version 4.2.1
mjanez Nov 4, 2024
8025dd9
Merge pull request #105 from mjanez/feature/2.10-update
mjanez Nov 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion .coveragerc
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
omit =
*/site-packages/*
*/python?.?/*
ckan/*
ckan/*
4 changes: 4 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
node_modules
build
dist
*.egg-info/
20 changes: 0 additions & 20 deletions .github/workflows/flake8.yml

This file was deleted.

75 changes: 50 additions & 25 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
name: schemingdcat-test
run-name: ${{ github.actor }} probando schemingdcat-test
on: [workflow_dispatch]
name: Tests
env:
COLUMNS: 120
on: [pull_request]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.10'
python-version: '3.9'
- name: Install requirements
run: pip install flake8 pycodestyle
- name: Check syntax
Expand All @@ -18,15 +19,23 @@ jobs:
needs: lint
strategy:
matrix:
ckan-version: ["2.10", 2.9]
include:
- ckan-version: "2.11"
ckan-image: "ckan/ckan-dev:2.11-py3.10"
- ckan-version: "2.10"
ckan-image: "ckan/ckan-dev:2.10-py3.10"
- ckan-version: "2.9"
ckan-image: "ckan/ckan-dev:2.9-py3.9"
fail-fast: false

name: CKAN ${{ matrix.ckan-version }}
runs-on: ubuntu-latest
container:
image: openknowledge/ckan-dev:${{ matrix.ckan-version }}
image: ${{ matrix.ckan-image }}
services:
solr:
image: ckan/ckan-solr:${{ matrix.ckan-version }}
# Custom image with spatial support and advanced schema
image: ghcr.io/mjanez/ckan-solr:2.10-solr9-spatial
postgres:
image: ckan/ckan-postgres-dev:${{ matrix.ckan-version }}
env:
Expand All @@ -35,7 +44,7 @@ jobs:
POSTGRES_DB: postgres
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
redis:
image: redis:3
image: redis:7-alpine
env:
CKAN_SQLALCHEMY_URL: postgresql://ckan_default:pass@postgres/ckan_test
CKAN_DATASTORE_WRITE_URL: postgresql://datastore_write:pass@postgres/datastore_test
Expand All @@ -44,25 +53,41 @@ jobs:
CKAN_REDIS_URL: redis://redis:6379/1

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v3
with:
repository: mjanez/ckanext-scheming
- name: Install requirements
- uses: actions/checkout@v4
- name: Install requirements (common)
run: |
pwd
echo $GITHUB_WORKSPACE
ls -al $GITHUB_WORKSPACE
pip install --upgrade pip
pip install -r requirements.txt
pip install -r dev-requirements.txt
pip install -e .
# Replace default path to CKAN core config file with the one on the container
sed -i -e 's/use = config:.*/use = config:\/srv\/app\/src\/ckan\/test-core.ini/' test.ini
sed -i -e 's/use = config:.*/use = config:\/srv\/app\/src\/ckan\/test-core.ini/' test_subclass.ini
- name: Install requirements (2.9)
run: |
pip install -U pytest-rerunfailures
if: ${{ matrix.ckan-version == '2.9' }}
- name: Setup other extensions
run: |
# Install ckanext-dcat
git clone https://github.com/mjanez/ckanext-dcat
pip install -e ckanext-dcat
pip install -r ckanext-dcat/requirements.txt

# Install ckanext-harvest
git clone https://github.com/ckan/ckanext-harvest
pip install -e ckanext-harvest
pip install -r ckanext-harvest/requirements.txt

# Install ckanext-scheming
git clone https://github.com/ckan/ckanext-scheming
pip install -e ckanext-scheming

# Install ckanext-fluent
git clone https://github.com/mjanez/ckanext-fluent
pip install -e ckanext-fluent

- name: Setup extension
run: |
pip install -r test-requirements.txt
ckan -c test.ini db init
- name: Run all tests
run: pytest --ckan-ini=test.ini --cov=ckanext.scheming ckanext/scheming/tests
- name: Run plugin subclassing tests
run: pytest --ckan-ini=test_subclass.ini ckanext/scheming/tests/test_dataset_display.py ckanext/scheming/tests/test_form.py::TestDatasetFormNew ckanext/scheming/tests/test_dataset_logic.py
ckan -c test.ini db pending-migrations --apply
- name: Run tests
run: pytest --log-cli-level=INFO --ckan-ini=test.ini --cov=ckanext.schemingdcat --cov-report=term-missing --cov-append --disable-warnings ckanext/schemingdcat/tests
18 changes: 18 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
.ropeproject
node_modules
**/node_modules/
bower_components
.vscode
.idea
*.log

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*.pyc
.pytest_cache

# C extensions
*.so
Expand Down Expand Up @@ -52,3 +57,16 @@ docs/_build/
ckanext/schemingdcat/i18n/tools/output/*
ckanext/schemingdcat/i18n/tools/input/*
ckanext/schemingdcat/i18n/tools/input/input.yaml

# custom harvesters
doc/harvesters/*_config.*.json
doc/harvesters/*_config.json
doc/harvesters/*.config.json
!doc/harvesters/ckan_template_config.json
!doc/harvesters/xls_template_config.json
!doc/harvesters/csw_template_config.json
!doc/harvesters/sql_template_config.json

# MKDocs
docs/tools/output
docs/tools/venv
Loading
Loading