| title | layout |
|---|---|
Unpublished datasets |
default |
Whilst data.gov.uk is best know for its catalogue of 'published' datasets, since September 2013 it has also included 'unpublished' datasets as well. These are datasets which exist but have not been made available publicly. In this way, data.gov.uk becomes an 'inventory' of data.
The reason for providing an inventory is to pursue a policy of 'open by default' and enable an open conversation about making specific datasets open or not. data.gov.uk has always has always provided a mechanism for the public to request data to be made open, and that is facilitated by knowing what data exists.
Clearly some many of the unpublished datasets are difficult or impossible to make public. For instance, much data about government employees needs to remain private (although details of senior civil servants, and counts of civil servants are available). However there are plenty of opportunities to open up data, and cataloguing them and providing a forum for both the public and the publisher reduces friction in this happening.
Please ensure that you are logged in and have been made an 'admin' for your publisher. If you do not see the blue "Publisher tools" button check out this guide. If you have the blue "Publisher tools" button but not the 'Manage unpublished datasets' link then you are an editor not an admin. In this case, contact data.gov.uk to request upgrade.
You can add to the catalogue your unpublished datasets from the management page for your publisher's unpublished datasets. To access this, first find your publisher page. When you log-in there is a link to your publisher page(s), or alternatively click on 'Data', 'Publishers', search for your publisher and then click on it. For example the 'Highways Agency' publisher page is at http://data.gov.uk/publisher/highways-agency
The management page is linked in the blue box:
The management page is relatively self-explanatory:
The suggested steps for reviewing and adding datasets are:
- Download the read-only dataset list to review your existing open data
- Download the inventory template
- Upload the completed inventory template
Once an unpublished dataset is listed on data.gov.uk, it can be changed by the publisher's editor or admin, in the same way as any dataset published using the form: on the dataset page click 'Edit':
and the edit form opens:
You can also change the unpublished dataset records on data.gov.uk by editing your existing spreadsheet of them and uploading it to data.gov.uk again. Assuming the titles of the datasets have not changed since the previous upload, the datasets on data.gov.uk will be edited with the changes in the spreadsheet. Any rows with a title which does not match will result in a new unpublished dataset being created.
If you delete rows from the spreadsheet and re-upload it, there will be no change to the equivalent dataset on data.gov.uk. To delete unpublished datasets, see below.
When a dataset is published then the record on data.gov.uk needs to reflect that.
In many cases you will simply want to convert the record into a published one, edited using the web form - this can be done on the dataset page by clicking
This will open the edit form for you to add the detail about the dataset - in particular you should complete the Licence and Data Files sections. When you save it (without any validation errors), it will at this point be in the 'published' state. However if you close the browser tab before saving it will remain 'unpublished'.
NB There are other circumstances when a straight conversion to a published record is not suitable. For example you might want to harvest the newly published record, rather than create it in a form, or there is not a 1:1 mapping of unpublished datasets to published datasets. In this case, we suggest you simply create the published datasets and then contact data.gov.uk to provide links to the unpublished and published datasets, pointing out the duplication and request the unpublished ones are deleted.
We anticipate a number of reasons why an unpublished dataset needs to be deleted, such as:
- The data has now been published and has been added to data.gov.uk in another way that using the 'convert to published' button.
- The unpublished dataset record on data.gov.uk is a duplicate of another (or is included in another).
- The unpublished dataset record was added to data.gov.uk by mistake - there was never any data.
- The data has been destroyed by the department.
The Data.gov.uk Team are happy to delete them for these reasons - please contact us providing links to each unpublished dataset and the specific reason for deletion.