GSoC 2024: Eye Tracking Algorithm Optimization Based on Low-Resolution Cameras (Final Project Submission)

.github/workflows/main.yml

@@ -2,13 +2,13 @@
 
 name: CI
 
-# Controls when the action will run. 
+# Controls when the action will run.
 on:
   # Triggers the workflow on push or pull request events but only for the main branch
   push:
-    branches: [ main ]
+    branches: [main]
   pull_request:
-    branches: [ main ]
+    branches: [main]
 
   # Allows you to run this workflow manually from the Actions tab
   workflow_dispatch:
@@ -19,7 +19,7 @@ jobs:
   build:
     # The type of runner that the job will run on
     runs-on: ubuntu-latest
-    
+
     # Steps represent a sequence of tasks that will be executed as part of the job
     steps:
       # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
@@ -29,5 +29,3 @@ jobs:
           heroku_api_key: ${{secrets.HEROKU_API_KEY}} # Located in GitHub secrets
           heroku_app_name: "web-eye-tracker-1204" # Must be unique in Heroku
           heroku_email: "[email protected]"
-
-

README.md

@@ -1,26 +1,26 @@
 # 👁️ Eye Lab: Gaze Tracker API
 
-Eye Lab is an open source tool to create eye tracking usability tests. It started as a final undergraduation work for Computer Engineering of student [Karine Pistili](https://www.linkedin.com/in/karine-pistili/) that created the first prototype. The idea is to evolve it to a more complete and useful tool with the help of the community.
+Eye Lab is an open-source tool to create eye-tracking usability tests. It started as a final undergraduate work for the Computer Engineering student [Karine Pistili](https://www.linkedin.com/in/karine-pistili/) who made the prototype. The idea is to evolve it into a more complete and useful tool with the community's help.
 
-The current version of the software allows users to create their usability sessions of an website, recording the webcam, screen and mouse movements and use this information to find out where the user has been looking into the screen by using heatmaps.
+The current version of the software allows users to create their usability sessions of a website, recording the webcam, screen, and mouse movements and use this information to find out where the user has been looking into the screen by using heatmaps.
 
 ## 👩‍💻 Setting up project locally
 
-The project consists of two parts, this repository contains the backend of the application and the frontend can be found [here](https://github.com/uramakilab/web-eye-tracker-front). Install it as well.
+The project consists of two parts, this repository contains the backend of the application, and the frontend can be found [here](https://github.com/uramakilab/web-eye-tracker-front). Install it as well to have the full application running.
 
 ### Prerequisites
 
-* [Python 3x](https://www.python.org/downloads/)
+- [Python 3x](https://www.python.org/downloads/)
 
 ### 1. Create virtual environment
 
-Before installing all dependencies and starting your Flask Server, it is better to create a python virtual environment. You can use the [venv package](https://docs.python.org/3/library/venv.html)
+Before installing all dependencies and starting your Flask Server, it is better to create a Python virtual environment. You can use the [venv package](https://docs.python.org/3/library/venv.html) to create a virtual environment. To create a new virtual environment, run the following command:
 
 ```
 python -m venv /path/to/new/virtual/environment
 ```
 
-Then activate your env. On windows for example you can activate with the script:
+Then activate your environment. On Windows for example you can activate with the script:
 
 ```
 name-of-event/Scripts/activate
@@ -40,12 +40,18 @@ pip install -r requirements.txt
 flask run
 ```
 
+## Contributors ✨
+
+The project is selected to be part of the [Google Summer of Code 2024](https://summerofcode.withgoogle.com/programs/2024/organizations/uramaki-lab) program, and [Vinícius Cavalcanti](https://github.com/hvini) is the main mentor of the project along with [Marc Gonzalez Capdevila](https://github.com/marcgc21), [Karine Pistili Rodrigues](https://github.com/KarinePistili). The active development of the project is being done by [Sitam Meur](https://github.com/sitamgithub-MSIT), selected as a GSoC'24 student for the project. Here are the project details in the [GSoC'24 website](https://summerofcode.withgoogle.com/programs/2024/projects/lEPzZg7S).
+
+To see the full list of contributions, check out the ahead commits of the "develop" branch concerning the "main" branch. Full logs of the project development can be found in the [Daily Work Progress](https://docs.google.com/document/d/1RjCnGjYYgPKvFUrN8hSjPX29aayWr6eEopeCN3QZwEQ/edit?usp=sharing) file. Hoping to see your name in the list of contributors soon! 🚀
+
 ## 🧑‍🤝‍🧑 Contributing
 
-Anyone is free to contribute to this project. Just do a pull request with your code and if it is all good we will accept it. You can also help us look for bugs, if you find anything create and issue.
+Anyone is free to contribute to this project. Just do a pull request with your code and if it is all good we will accept it. You can also help us look for bugs if you find anything that creates an issue.
 
 ## 📃 License
 
-This software is under the [MIT License](https://opensource.org/licenses/MIT). 
+This software is under the [MIT License](https://opensource.org/licenses/MIT).
 
 Copyright 2021 Uramaki Lab

app/main.py

@@ -1,11 +1,16 @@
+# Necessary imports
 from flask import Flask, request, Response
 from flask_cors import CORS
+
+# Local imports from app
 from app.routes import session as session_route
-import os
 
+
+# Initialize Flask app and enable CORS
 app = Flask(__name__)
 CORS(app)
 
+
 # @app.route('/', methods=['GET'])
 # def welcome():
 #     return Response(f'Welcome to EyeLab!', status=200, mimetype='application/json')
@@ -51,8 +56,18 @@
 #     return Response('Invalid request method for route', status=405, mimetype='application/json')
 
 
-@app.route('/api/session/calib_validation', methods=['POST'])
+# Route for validating calibration
+@app.route("/api/session/calib_validation", methods=["POST"])
 def calib_validation():
-    if request.method == 'POST':
+    """
+    Validates the calibration request.
+
+    Returns:
+        If the request method is 'POST', it calls the `calib_results` function from the `session_route` module.
+        Otherwise, it returns a `Response` object with an error message and status code 405.
+    """
+    if request.method == "POST":
         return session_route.calib_results()
-    return Response('Invalid request method for route', status=405, mimetype='application/json')
+    return Response(
+        "Invalid request method for route", status=405, mimetype="application/json"
+    )

app/models/session.py

@@ -1,5 +1,35 @@
 class Session:
-    def __init__(self, id, title, description, user_id, created_date, website_url, screen_record_url, webcam_record_url, heatmap_url, calib_points, iris_points):
+    """
+    Represents a session in the eye tracking application.
+
+    Attributes:
+        - id (int): The unique identifier of the session.
+        - title (str): The title of the session.
+        - description (str): The description of the session.
+        - user_id (int): The user ID associated with the session.
+        - created_date (datetime): The date and time when the session was created.
+        - website_url (str): The URL of the website being tracked.
+        - screen_record_url (str): The URL of the screen recording for the session.
+        - webcam_record_url (str): The URL of the webcam recording for the session.
+        - heatmap_url (str): The URL of the heatmap image for the session.
+        - calib_points (list): The calibration points used in the session.
+        - iris_points (list): The iris tracking points recorded in the session.
+    """
+
+    def __init__(
+        self,
+        id,
+        title,
+        description,
+        user_id,
+        created_date,
+        website_url,
+        screen_record_url,
+        webcam_record_url,
+        heatmap_url,
+        calib_points,
+        iris_points,
+    ):
         self.id = id
         self.title = title
         self.description = description
@@ -13,16 +43,22 @@ def __init__(self, id, title, description, user_id, created_date, website_url, s
         self.iris_points = iris_points
 
     def to_dict(self):
+        """
+        Converts the session object to a dictionary.
+
+        Returns:
+            dict: A dictionary representation of the session object.
+        """
         return {
-            u'id': self.id,
-            u'title': self.title,
-            u'description': self.description,
-            u'user_id': self.user_id,
-            u'created_date': self.created_date,
-            u'website_url': self.website_url,
-            u'screen_record_url': self.screen_record_url,
-            u'webcam_record_url': self.webcam_record_url,
-            u'heatmap_url': self.heatmap_url,
-            u'callib_points': self.calib_points,
-            u'iris_points': self.iris_points
-        }
+            "id": self.id,
+            "title": self.title,
+            "description": self.description,
+            "user_id": self.user_id,
+            "created_date": self.created_date,
+            "website_url": self.website_url,
+            "screen_record_url": self.screen_record_url,
+            "webcam_record_url": self.webcam_record_url,
+            "heatmap_url": self.heatmap_url,
+            "callib_points": self.calib_points,
+            "iris_points": self.iris_points,
+        }

app/routes/session.py

@@ -1,18 +1,26 @@
-from flask import Flask, request, Response, send_file
-from app.services.storage import save_file_locally
-from app.models.session import Session
-#from app.services import database as db
-from app.services import gaze_tracker
+# Necesary imports
+import os
+import re
 import time
 import json
 import csv
+
 from pathlib import Path
-import os
-import re
+from flask import Flask, request, Response, send_file
+
+# Local imports from app
+from app.services.storage import save_file_locally
+from app.models.session import Session
+
+# from app.services import database as db
+from app.services import gaze_tracker
 
-ALLOWED_EXTENSIONS = {'txt', 'webm'}
-COLLECTION_NAME = u'session'
 
+# Constants
+ALLOWED_EXTENSIONS = {"txt", "webm"}
+COLLECTION_NAME = "session"
+
+# Initialize Flask app
 app = Flask(__name__)
 
 
@@ -139,49 +147,85 @@
 
 
 def calib_results():
-    file_name = json.loads(request.form['file_name'])
-    fixed_points = json.loads(request.form['fixed_circle_iris_points'])
-    calib_points = json.loads(request.form['calib_circle_iris_points'])
-    screen_height = json.loads(request.form['screen_height'])
-    screen_width = json.loads(request.form['screen_width'])
-    k = json.loads(request.form['k'])
+    """
+    Generate calibration results.
+
+    This function generates calibration results based on the provided form data.
+    It saves the calibration points to a CSV file. Then, it uses the gaze_tracker module to predict the calibration results.
+
+    Returns:
+        Response: A JSON response containing the calibration results.
+
+    Raises:
+        IOError: If there is an error while writing to the CSV files.
+    """
+    # Get form data from request
+    file_name = json.loads(request.form["file_name"])
+    fixed_points = json.loads(request.form["fixed_circle_iris_points"])
+    calib_points = json.loads(request.form["calib_circle_iris_points"])
+    screen_height = json.loads(request.form["screen_height"])
+    screen_width = json.loads(request.form["screen_width"])
+    k = json.loads(request.form["k"])
+    model = json.loads(request.form["model"])
 
     # Generate csv dataset of calibration points
     os.makedirs(
-        f'{Path().absolute()}/app/services/calib_validation/csv/data/', exist_ok=True)
-    calib_csv_file = f'{Path().absolute()}/app/services/calib_validation/csv/data/{file_name}_fixed_train_data.csv'
-    csv_columns = ['left_iris_x', 'left_iris_y',
-                   'right_iris_x', 'right_iris_y', 'point_x', 'point_y', 'screen_height', 'screen_width']
+        f"{Path().absolute()}/app/services/calib_validation/csv/data/", exist_ok=True
+    )
+
+    # Generate csv of calibration points with following columns
+    calib_csv_file = f"{Path().absolute()}/app/services/calib_validation/csv/data/{file_name}_fixed_train_data.csv"
+    csv_columns = [
+        "left_iris_x",
+        "left_iris_y",
+        "right_iris_x",
+        "right_iris_y",
+        "point_x",
+        "point_y",
+        "screen_height",
+        "screen_width",
+    ]
+
+    # Save calibration points to CSV file
     try:
-        with open(calib_csv_file, 'w') as csvfile:
+        # Open CSV file
+        with open(calib_csv_file, "w") as csvfile:
             writer = csv.DictWriter(csvfile, fieldnames=csv_columns)
             writer.writeheader()
+
+            # Write calibration points to CSV file
             for data in fixed_points:
-                data['screen_height'] = screen_height
-                data['screen_width'] = screen_width
+                data["screen_height"] = screen_height
+                data["screen_width"] = screen_width
                 writer.writerow(data)
+
+    # Handle I/O error
     except IOError:
         print("I/O error")
 
     # Generate csv of iris points of session
     os.makedirs(
-        f'{Path().absolute()}/app/services/calib_validation/csv/data/', exist_ok=True)
-    predict_csv_file = f'{Path().absolute()}/app/services/calib_validation/csv/data/{file_name}_predict_train_data.csv'
-    csv_columns = ['left_iris_x', 'left_iris_y',
-                   'right_iris_x', 'right_iris_y']
+        f"{Path().absolute()}/app/services/calib_validation/csv/data/", exist_ok=True
+    )
+    predict_csv_file = f"{Path().absolute()}/app/services/calib_validation/csv/data/{file_name}_predict_train_data.csv"
+    csv_columns = ["left_iris_x", "left_iris_y", "right_iris_x", "right_iris_y"]
     try:
-        with open(predict_csv_file, 'w') as csvfile:
+        with open(predict_csv_file, "w") as csvfile:
             writer = csv.DictWriter(csvfile, fieldnames=csv_columns)
             writer.writeheader()
             for data in calib_points:
+                # print(data)
                 writer.writerow(data)
     except IOError:
         print("I/O error")
 
     # data = gaze_tracker.train_to_validate_calib(calib_csv_file, predict_csv_file)
-    data = gaze_tracker.predict(calib_csv_file, calib_csv_file, k)
 
-    return Response(json.dumps(data), status=200, mimetype='application/json')
+    # Predict calibration results
+    data = gaze_tracker.predict(calib_csv_file, k, model_X=model, model_Y=model)
+
+    # Return calibration results
+    return Response(json.dumps(data), status=200, mimetype="application/json")
 
 
 # def session_results():