Skip to content

Commit

Permalink
site update
Browse files Browse the repository at this point in the history
  • Loading branch information
@lmkr
lmkr committed Sep 12, 2023
1 parent 12422ef commit cb3b993
Show file tree
Hide file tree
Showing 7 changed files with 68 additions and 34 deletions.
24 changes: 23 additions & 1 deletion architecture/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -189,9 +189,31 @@ <h1 class="js-title">Architecture</h1>


<p class="lead">
<img src="../soparchitecture.png" alt="">
The architecture of the SmartOcean platform is based on a set of microservices providing the ability to consume and provide data from both internal and external systems.
</p>

<p>The figure below shows the overall architecture of the SmartOcean platform comprised of the following services:</p>
<ul>
<li>Data space service: providing longer-term access to data</li>
<li>Messaging service: providing access to real-time data streams</li>
<li>Edge integration service: providing data and control integration with the underwater sensor networks</li>
<li>Security service: providing authentication and authorization services</li>
<li>Data quality service: providing data quality control and outlier detection</li>
<li>Monitoring service: providing key performance indicators for the platform</li>
</ul>
<p><img src="../soparchitecture.png" alt=""></p>
<p>The SmartOcean platform architecture is based on a comprehensive interview study with stakeholders and domain expert in order to solicit requirements. The architecture is documented in the following publications:</p>
<ul>
<li>
<p>N-T. Nguyen, R. Heldal, K. Lima, T.D. Oyetoyan, P. Pelliccione, L.M. Kristensen, K.W. Høydal, P.A Reiersgaard, Y. Kvinnsland: <em>Engineering Challenges of Stationary Wireless Smart Ocean Observation Systems</em>. IEEE Internet Things J. 10(16): 14712-14724 (2023)</p>
</li>
<li>
<p>R. Heldal, L.M. Kristensen, K. Lima, T. D. Oyetoyan, N-T. Nguyen: <em>Towards a Formal and Executable Software Architecture Specification of the Smart Ocean Data Service Platform</em>. PNSE@Petri Nets 2023: 110-125, 2022.</p>
</li>
<li>
<p>K. Lima, N-T. Nguyen, R. Heldal, E. Knauss, T.D. Oyetoyan, P. Pelliccione, L.M. Kristensen: <em>Marine Data Sharing: Challenges, Technology Drivers and Quality Attributes</em>. PROFES 2022: 124-140</p>
</li>
</ul>



Expand Down
22 changes: 12 additions & 10 deletions dataconsumers/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,12 @@

</div>
<div class="docs-toc large order-lg-2 order-md-0 order-xs-1 col-12 col-lg-2 col-xl-2 position-sticky border-left"><div class="docs-toc">
<nav id="TableOfContents"></nav>
<nav id="TableOfContents">
<ul>
<li><a href="#smartocean-austevoll-data-sethttpmetadatanmdcnometadata-apilandingpagef4ef202da4c6dc873c7f67140a430ef9"><a href="http://metadata.nmdc.no/metadata-api/landingpage/f4ef202da4c6dc873c7f67140a430ef9">SmartOcean Austevoll Data Set</a></a></li>
<li><a href="#sfi-smartocean-data-exploration-portalhttpssmartoceanwebnorcecloudviewdefault"><a href="https://smartocean.web.norce.cloud/#/view/default">SFI SmartOcean Data Exploration Portal</a></a></li>
</ul>
</nav>
</div>
</div>
<div class="main col-12 order-1 col-md-9 col-lg-10 col-xl-8 py-3">
Expand All @@ -197,18 +202,15 @@ <h1 class="js-title">Data consumers</h1>
<li><a href="https://smartoceanplatform.github.io/dataspaces/">REST API of the data space service</a></li>
<li><a href="https://smartoceanplatform.github.io/messaging/">MQTT core messaging service</a></li>
</ul>
<p>The ThingSpeak channel below consumes the HVL Virtual Sensor Hub and has been setup for demonstration purposes</p>
<p>The ThingSpeak channel below consumes the HVL Virtual Sensor Hub and has been setup for public demonstration purposes</p>
<ul>
<li><a href="https://thingspeak.com/channels/2129424">Virtual Sensor Hub ThingSpeak Channel</a></li>
</ul>
<p>Data streams from the platform are also being consumed by and made available via the <a href="https://www.nmdc.no/nmdc">Norwegian Marine Data Centre</a></p>
<ul>
<li>SmartOcean Austevoll Data Set (LINK IS PENDING)</li>
</ul>
<p>and can be inspected via the Norce Enlighten Portal</p>
<ul>
<li><a href="https://smartocean.web.norce.cloud/#/view/default">SFI SmartOcean Data Exploration Portal</a></li>
</ul>
<p>Data streams from the platform are also being consumed by and made available via the <a href="https://www.nmdc.no/nmdc">Norwegian Marine Data Centre</a> and via the Norce Enlighten Portal</p>
<h2 id="smartocean-austevoll-data-sethttpmetadatanmdcnometadata-apilandingpagef4ef202da4c6dc873c7f67140a430ef9"><a href="http://metadata.nmdc.no/metadata-api/landingpage/f4ef202da4c6dc873c7f67140a430ef9">SmartOcean Austevoll Data Set</a></h2>
<iframe width="850" height="460" style="border: 1px solid #cccccc;" src="http://metadata.nmdc.no/metadata-api/landingpage/f4ef202da4c6dc873c7f67140a430ef9"></iframe>
<h2 id="sfi-smartocean-data-exploration-portalhttpssmartoceanwebnorcecloudviewdefault"><a href="https://smartocean.web.norce.cloud/#/view/default">SFI SmartOcean Data Exploration Portal</a></h2>
<iframe width="850" height="460" style="border: 1px solid #cccccc;" src="https://smartocean.web.norce.cloud/#/view/default"></iframe>



Expand Down
18 changes: 9 additions & 9 deletions dataproviders/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -189,19 +189,19 @@ <h1 class="js-title">Data providers</h1>


<p class="lead">
<p>Data providers are systems and applications that actively deliver data to the Smart Ocean platform.</p>
<p>Data is currently being provided to the platform from the <a href="https://www.hi.no/en/hi/laboratories/austevoll-research-station">Austevoll Research Station</a> of the <a href="https://www.hi.no/hi/en">Institute of Marine Research</a> where SmartOcean Pilot Demonstrator 1 is deployed.</p>
<img src="../pilotdemonstrator1.png" height="300">
<p></p>
Data providers are systems and applications that actively deliver marine data-sets and data streams to the Smart Ocean platform.
</p>

<p>Data is currently being provided to the platform from the <a href="https://www.hi.no/en/hi/laboratories/austevoll-research-station">Austevoll Research Station</a> of the <a href="https://www.hi.no/hi/en">Institute of Marine Research</a> where the SmartOcean Pilot Demonstrator 1 has been deployed.</p>
<p><img src="../pilotdemonstrator1_austevoll.png" alt=""></p>
<p>The pilot demonstrator encompass underwater sensor hubs and nodes with underwater wireless communication from</p>
<ul>
<li><a href="https://www.aanderaa.com/profiling-current-meter">Aanderaa Data Instruments</a></li>
<li><a href="https://www.aanderaa.com/profiling-current-meter">Aanderaa Data Instruments</a> (AADI/Xylem)</li>
<li><a href="https://wsense.it">W-sense</a></li>
</ul>
<p>Data is being provided to the platform via the <a href="https://smartoceanplatform.github.io/messaging/">ingestion messaging service</a>.</p>

</p>

<p>Data is being provided to the platform via the <a href="https://smartoceanplatform.github.io/messaging/">ingestion messaging service</a> as illustrated in the figure below</p>
<p><img src="../providers.png" alt=""></p>
<p>The data ingestion service does not provided any constrained on the format in which data is being provided in order to support integration of a wide range of data sources.</p>



Expand Down
8 changes: 4 additions & 4 deletions dataqualitycontrol/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -189,12 +189,12 @@ <h1 class="js-title">Data quality</h1>


<p class="lead">
<p>In-situ marine data is prone to error due to several factors. Underwater sensors may produce a lot of anomalies when powered by low-batteries. Biofouling is an accumulation of microorganisms, plants, algae, or small animals on underwater sensors, resulting in unexpected measurement results. If low-quality data is used in decision-making processes, the results will be misleading or suboptimal; thus, data quality control (DQC) must be done to ensure data reliability.</p>
<p>For now, DQC activities predominantly relies on manual checks. Thanks to today&rsquo;s advanced sensory technologies, vast amounts of data are collected on a daily basis. Manual DQC can be time-consuming and lead to significant delays in data publication. To address this challenge and speed up the DQC process, we are developing an automated DQC framework called Adaptive Anomaly Detector (AdapAD) based on Artificial Intelligence (AI), specifically unsupervised anomaly detection.</p>
<p>The framework aims to detect anomalous measurements from real-time data, which support data validators and data consumers to assess the quality of data for further usage.</p>


</p>

<p>In-situ marine data is prone to error due to several factors. Underwater sensors may produce a lot of anomalies when powered by low-batteries. Biofouling is an accumulation of microorganisms, plants, algae, or small animals on underwater sensors, resulting in unexpected measurement results. If low-quality data is used in decision-making processes, the results will be misleading or suboptimal; thus, data quality control (DQC) must be done to ensure data reliability.</p>
<p>For now, DQC activities predominantly relies on manual checks. Thanks to today&rsquo;s advanced sensory technologies, vast amounts of data are collected on a daily basis. Manual DQC can be time-consuming and lead to significant delays in data publication. To address this challenge and speed up the DQC process, we are developing an automated DQC framework called Adaptive Anomaly Detector (AdapAD) based on Artificial Intelligence (AI), specifically unsupervised anomaly detection.</p>
<p>The framework aims to detect anomalous measurements from real-time data, which support data validators and data consumers to assess the quality of data for further usage.</p>



Expand Down
10 changes: 5 additions & 5 deletions index.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"title": "Messaging",
"tags": null,
"description": "",
"content": "The messaging service is based on the publish-subscribe paradigm and the MQTT protocol. In this paradigm, data providers publish data topics while data consumers subscribe to topic in order to receive data published on the topic.\nThe messaging service is comprised of two HiveMQ MQTT messaging clusters\n A Data Ingestion Messaging Service for delivering data to the platform A Core Messaging Messaging Service for data transformation, forwarding and consumption The data ingestion service requires authentication for data provisioning with the following topic structure\nsmartocean/\u0026lt;domain\u0026gt;/\u0026lt;site\u0026gt;/\u0026lt;subsite\u0026gt;/\u0026lt;node\u0026gt; Currently the following topics are in used corresponding to the data sources of SFI SmartOcean pilot demonstrator 1\nsmartocean/pilotdemo_0001/austevoll/north/seaguardII_0001 smartocean/pilotdemo_0001/austevoll/south/seaguardII_0002 smartocean/pilotdemo_0001/austevoll/wsense/node1 smartocean/testing/virtualsensorhub "
"content": "The messaging service of the SmartOcean platform supports publishing data and subscribing to real-time data streams The messaging service is based on the publish-subscribe paradigm and the MQTT protocol. In this paradigm, data providers publish data topics while data consumers subscribe to topic in order to receive data published on the topic.\nThe messaging service is comprised of two HiveMQ MQTT messaging clusters\n A Data Ingestion Messaging Service for delivering data to the platform A Core Messaging Messaging Service for data transformation, forwarding and consumption as illustrated in the figure below. The data ingestion service receives data in formats determined by the data sources and a transformation to the SmartOcean data format is performed prior to publishing the data on the core messaging service for further processing.\nThe data ingestion service requires authentication for data provisioning with the following topic structure\nsmartocean/\u0026lt;domain\u0026gt;/\u0026lt;site\u0026gt;/\u0026lt;subsite\u0026gt;/\u0026lt;node\u0026gt; The data transformation component published the transformed data on the corresponding topic:\nsmartocean/\u0026lt;domain\u0026gt;/\u0026lt;site\u0026gt;/\u0026lt;subsite\u0026gt;/\u0026lt;node\u0026gt;/raw Currently the following topics are in used corresponding to the data sources of SFI SmartOcean pilot demonstrator 1\nsmartocean/pilotdemo_0001/austevoll/north/seaguardII_0001 smartocean/pilotdemo_0001/austevoll/south/seaguardII_0002 smartocean/pilotdemo_0001/austevoll/wsense/node1 smartocean/testing/virtualsensorhub "
},
{
"uri": "https://smartoceanplatform.github.io/edgeintegration/",
Expand All @@ -39,21 +39,21 @@
"title": "Data consumers",
"tags": null,
"description": "",
"content": "Data consumers are systems and applications that actively use data services from the SmartOcean platform. Data streams from the SmartOcean platform can be consumed via the:\n REST API of the data space service MQTT core messaging service The ThingSpeak channel below consumes the HVL Virtual Sensor Hub and has been setup for demonstration purposes\n Virtual Sensor Hub ThingSpeak Channel Data streams from the platform are also being consumed by and made available via the Norwegian Marine Data Centre\n SmartOcean Austevoll Data Set (LINK IS PENDING) and can be inspected via the Norce Enlighten Portal\n SFI SmartOcean Data Exploration Portal "
"content": "Data consumers are systems and applications that actively use data services from the SmartOcean platform. Data streams from the SmartOcean platform can be consumed via the:\n REST API of the data space service MQTT core messaging service The ThingSpeak channel below consumes the HVL Virtual Sensor Hub and has been setup for public demonstration purposes\n Virtual Sensor Hub ThingSpeak Channel Data streams from the platform are also being consumed by and made available via the Norwegian Marine Data Centre and via the Norce Enlighten Portal\nSmartOcean Austevoll Data Set SFI SmartOcean Data Exploration Portal "
},
{
"uri": "https://smartoceanplatform.github.io/dataproviders/",
"title": "Data providers",
"tags": null,
"description": "",
"content": "Data providers are systems and applications that actively deliver data to the Smart Ocean platform.\nData is currently being provided to the platform from the Austevoll Research Station of the Institute of Marine Research where SmartOcean Pilot Demonstrator 1 is deployed.\n\nThe pilot demonstrator encompass underwater sensor hubs and nodes with underwater wireless communication from\n Aanderaa Data Instruments W-sense Data is being provided to the platform via the ingestion messaging service.\n"
"content": "Data providers are systems and applications that actively deliver marine data-sets and data streams to the Smart Ocean platform. Data is currently being provided to the platform from the Austevoll Research Station of the Institute of Marine Research where the SmartOcean Pilot Demonstrator 1 has been deployed.\nThe pilot demonstrator encompass underwater sensor hubs and nodes with underwater wireless communication from\n Aanderaa Data Instruments (AADI/Xylem) W-sense Data is being provided to the platform via the ingestion messaging service as illustrated in the figure below\nThe data ingestion service does not provided any constrained on the format in which data is being provided in order to support integration of a wide range of data sources.\n"
},
{
"uri": "https://smartoceanplatform.github.io/interoperability/",
"title": "Interoperability",
"tags": null,
"description": "",
"content": "Under construction "
"content": "Interoperability is a central aspect of the SmartOcean platform which provided data provision and consumption via standardised REST and MQTT cloud APIs, and data and meta-data formats for service integration Work is currently ongoing on standardising a data- and meta-data format for the SmartOcean platform to be used both internally in the platform and for exposing data. The figure below shows the current basic data model being used by the SmartOcean platform.\nAn implementation of the data format in Python is available for consortium members:\n https://github.com/smartoceanplatform/datamodels "
},
{
"uri": "https://smartoceanplatform.github.io/security/",
Expand Down Expand Up @@ -81,7 +81,7 @@
"title": "Architecture",
"tags": null,
"description": "",
"content": ""
"content": "The architecture of the SmartOcean platform is based on a set of microservices providing the ability to consume and provide data from both internal and external systems. The figure below shows the overall architecture of the SmartOcean platform comprised of the following services:\n Data space service: providing longer-term access to data Messaging service: providing access to real-time data streams Edge integration service: providing data and control integration with the underwater sensor networks Security service: providing authentication and authorization services Data quality service: providing data quality control and outlier detection Monitoring service: providing key performance indicators for the platform The SmartOcean platform architecture is based on a comprehensive interview study with stakeholders and domain expert in order to solicit requirements. The architecture is documented in the following publications:\n N-T. Nguyen, R. Heldal, K. Lima, T.D. Oyetoyan, P. Pelliccione, L.M. Kristensen, K.W. Høydal, P.A Reiersgaard, Y. Kvinnsland: Engineering Challenges of Stationary Wireless Smart Ocean Observation Systems. IEEE Internet Things J. 10(16): 14712-14724 (2023)\n R. Heldal, L.M. Kristensen, K. Lima, T. D. Oyetoyan, N-T. Nguyen: Towards a Formal and Executable Software Architecture Specification of the Smart Ocean Data Service Platform. PNSE@Petri Nets 2023: 110-125, 2022.\n K. Lima, N-T. Nguyen, R. Heldal, E. Knauss, T.D. Oyetoyan, P. Pelliccione, L.M. Kristensen: Marine Data Sharing: Challenges, Technology Drivers and Quality Attributes. PROFES 2022: 124-140\n "
},
{
"uri": "https://smartoceanplatform.github.io/categories/",
Expand Down
8 changes: 7 additions & 1 deletion interoperability/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -189,9 +189,15 @@ <h1 class="js-title">Interoperability</h1>


<p class="lead">
Under construction
Interoperability is a central aspect of the SmartOcean platform which provided data provision and consumption via standardised REST and MQTT cloud APIs, and data and meta-data formats for service integration
</p>

<p>Work is currently ongoing on standardising a data- and meta-data format for the SmartOcean platform to be used both internally in the platform and for exposing data. The figure below shows the current basic data model being used by the SmartOcean platform.</p>
<p><img src="../sodataformat.png" alt=""></p>
<p>An implementation of the data format in Python is available for consortium members:</p>
<ul>
<li><a href="https://github.com/smartoceanplatform/datamodels">https://github.com/smartoceanplatform/datamodels</a></li>
</ul>



Expand Down
Loading

0 comments on commit cb3b993

Please sign in to comment.