Automated development update from jacobmartinez3d

README.md

@@ -2,10 +2,10 @@
   <img src="media/magla_banner.png">
 </p>
 
+![Python package](https://github.com/magnetic-lab/magla/workflows/Python%20package/badge.svg?branch=master)
 ![stability-experimental](https://img.shields.io/badge/stability-experimental-orange.svg)
 
 ## Magnetic-Lab Pipeline API for Content Creators
-
 Magla is an effort to bring the magic of large-scale professional visual effects pipelines to small-scale studios and freelancers - for free. Magla features a backend designed to re-enforce the contextual relationships between things in a visual effects pipeline - a philosophy which is at the core of Magla's design. The idea is that with any given `MaglaEntity` one can traverse through all the related `entities` as they exist in the DB. This is achieved with a `Postgres` + `SQLAlchemy` combination allowing for an excellent object-oriented interface with powerful SQL queries and relationships behind it.
 
 #### Example:
@@ -22,14 +22,15 @@ project_settings = user.assignments[-1].shot_version.shot.project.settings
 project_settings = user.assignments[-1].project.settings
 ```
 Comparing the above examples to the diagrom below and you can see the connective routes that can be traversed based on Magla's schema relationships:
+
 <img src="media/ERD.png">
 
 ### [OpenTimelineIO](https://github.com/PixarAnimationStudios/OpenTimelineIO)-centric design
-In the heat of production there is always a consistent demand for creating, viewing, and generally altering edits in various ways and in various contexts, for all kinds of reasons. This is the reason for another core philosophy of Magla, which is that timelines and edits should be the driving force of the pipeline. In simple terms the goal in a Magla production-environment is to create, mutate, and move data into a refined form: an edit(s) capable of outputting various formats demanded by the client.
+In the heat of production there is always a consistent demand for creating, viewing, and generally altering edits in various ways and in various contexts, for all kinds of reasons. This is the reason for another core philosophy of Magla, which is that timelines and edits should be the driving force of the pipeline.
 
 In Magla, timelines can be requested, and then dynamically generated on the fly using your production data. This will enable superior features development and automation, as well as hopefully break some shackles and give the idea of an edit more of an expressionistic, non-binding and ultimitely, more creative feeling. 
 
-`MaglaProject`, `MaglaShot`, and `MaglaShotVersion` objects all include companion `opentimelineio.schema` objects which are mirror representations of eachother. The `opentimelineio` objects are saved in `JSON` form in the DB.
+`MaglaProject`, `MaglaShot`, and `MaglaShotVersion` objects all include companion `opentimelineio.schema` objects which are mirror representations of eachother. The `opentimelineio` objects are saved in `JSONB` form in the DB.
 
 Breakdown of `MaglaEntity` types and their associated `opentimelineio.schema` types:
 - `Project` <--> `opentimelineio.schema.Timeline`
@@ -38,6 +39,34 @@ Breakdown of `MaglaEntity` types and their associated `opentimelineio.schema` ty
 
 in the Magla ecosystem `ShotVersion`'s are considered sacred and only one can be current at any given time, even new assignments result in new versions. For this reson they are used as the actual `ExternalReference` of the shot `Clip` -  so only the latest versions of shots are used as meda references. Each time you instantiate a `MaglaProject` object it builds its otio data off of current production data and thus is always up-to-date and **requires no actual timeline file to  be archived on disk or kept track of**.
 
+## Getting Started
+You will need to first set the following environment variables required for `magla` to function:
+
+- `MAGLA_DB_DATA_DIR` <-- this is where your `sqlite` db will be written to
+- `MAGLA_DB_NAME` <-- name of your DB
+- `MAGLA_MACHINE_CONFIG_DIR` <-- this directory holds information about the current machine needed by `magla`
+
+Linux:
+```bash
+export MAGLA_DB_DATA_DIR=/path/to/magla_data_dir
+export MAGLA_DB_NAME=magla
+export MAGLA_MACHINE_CONFIG_DIR=/path/to/magla_machine_dir
+```
+
+Windows:
+```cmd
+SET MAGLA_DB_DATA_DIR="<drive>:\\path\to\magla_data_dir"
+SET MAGLA_DB_NAME="magla"
+SET MAGLA_MACHINE_CONFIG_DIR="<drive>:\\path\to\magla_machine_dir"
+```
+
+### Installing
+```bash
+git clone https://github.com/magnetic-lab/magla.git
+cd magla
+pip install .
+```
+
 ### Example Initial Setup
 All creation and deletion methods are in `magla.Root`, so this is primarily a demonstration of
 using the creation methods in the optimal order.
@@ -73,7 +102,7 @@ facility = r.create_facility("test_facility",
 ```
 The above creates a new `Postgres` column in the 'facilities' table and returns a `MaglaFacility` object pre-populated with data in the '<MaglaEntity>.data' property.
 
-Project settings are sent in as a dictionary which is stored as `JSON` in `Postgres`. At runtime a `MaglaEntity` object gets injected and Python's native string formatting can be used to access the object's relationships and attributes for custom naming.
+Project settings are sent in as a dictionary which is stored as `JSONB` in `Postgres`. At runtime a `MaglaEntity` object gets injected and Python's native string formatting can be used to access the object's relationships and attributes for custom naming.
 ```python
 # Create 2D settings template
 # a custom creation method doesn't exist for this entity type so the 'create' method is used directly.
@@ -179,6 +208,25 @@ t.build(test_project.shots)
 t.otio.to_json_file("test_project.json")
 ```
 
+### Development Setup:
+Running tests require a `MAGLA_TEST_DIR` environment varable pointing to a directory containing `seed_data.yaml` and `test_project.otio` files. Initially you can set this to the included `magla/tests` directory.
+```bash
+export MAGLA_TEST_DIR=/path/to/magla/tests
+```
+
+#### Installing in dev mode:
+```bash
+git clone https://github.com/magnetic-lab/magla.git
+cd magla
+pip install .[dev]
+```
+
+#### Running coverage report and tests:
+```bash
+coverage run --source magla -m pytest -v
+coverage report
+```
+
 ## Magla Roadmap
 <p>
 <img src="media/magla.png">

magla/core/assignment.py

@@ -1,3 +1,4 @@
+# -*- coding: utf-8 -*-
 """Assignments connect `MaglaUser` to `MaglaShotVersion`. 
 
 In the `magla` ecosystem versions are considered sacred and can never be assigned to multiple

magla/core/directory.py

@@ -137,7 +137,7 @@ def tree(self):
         Returns
         -------
         dict
-            Postgres column tree (JSON)
+            Postgres column tree (JSONB)
         """
         return self.data.tree
 
@@ -148,7 +148,7 @@ def bookmarks(self):
         Returns
         -------
         dict
-            Postgres column bookmarks (JSON)
+            Postgres column bookmarks (JSONB)
         """
         return self.data.bookmarks
 

magla/core/flask_utils.py

@@ -0,0 +1,15 @@
+class FlaskLoginUser(object):
+    def __init__(self):
+        self._authenticated = False
+
+    def is_authenticated(self):
+        return self._authenticated
+
+    def is_active(self):
+        return self.active
+
+    def is_anonumous(self):
+        return False
+
+    def get_id(self):
+        return str(self.id).encode("utf-8")

magla/core/project.py

@@ -21,7 +21,7 @@ class MaglaProject(MaglaEntity):
     A project consists of:
         - A root directory located on a machine within the current facility
         - A collection of child `MaglaShot`, `MaglaShotVersion`, `MaglaToolConfig` entities
-        - An `opentimelineio.schema.Timeline` object which persists in the backend as `JSON`
+        - An `opentimelineio.schema.Timeline` object which persists in the backend as `JSONB`
         - User-defined settings containing python string-formatting tokens
 
     Defining project settings:

magla/core/root.py

@@ -5,6 +5,7 @@
 creation methods for convenience.
 """
 import os
+import psycopg2
 import re
 import shutil
 import sys
@@ -593,7 +594,7 @@ def create_user(self, data):
         new_user = self.create(MaglaUser, data)
         # create default home directory for user
         machine = MaglaMachine()
-        new_directory = self.create(MaglaDirectory, {
+        self.create(MaglaDirectory, {
             "machine_id": machine.id,
             "user_id": new_user.id,
             "label": "default",
@@ -604,7 +605,7 @@ def create_user(self, data):
                 "id": new_user.id,
                 "machine_id": machine.id
             })
-        except EntityAlreadyExistsError:
+        except:
             pass
         return new_user
 

magla/core/user.py

@@ -2,6 +2,7 @@
 import getpass
 
 from ..db.user import User
+from .flask_utils import FlaskLoginUser
 from .entity import MaglaEntity
 from .errors import MaglaError
 
@@ -10,7 +11,7 @@ class MaglaUserError(MaglaError):
     """An error accured preventing MaglaUser to continue."""
 
 
-class MaglaUser(MaglaEntity):
+class MaglaUser(MaglaEntity, FlaskLoginUser):
     """Provide interface to user details and privileges."""
     __schema__ = User
 
@@ -83,6 +84,17 @@ def email(self):
             email of this user
         """
         return self.data.email
+
+    @property
+    def active(self):
+        """Retrieve active flag from data.
+
+        Returns
+        -------
+        str
+            flag for whether user is currently active
+        """
+        return self.data.active
 
     # SQAlchemy relationship back-references
     @property

magla/db/dependency.py

@@ -1,5 +1,5 @@
 from sqlalchemy import Column, Integer, String
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 
 from ..db.orm import MaglaORM
 
@@ -11,4 +11,4 @@ class Dependency(MaglaORM._Base):
 
     id = Column(Integer, primary_key=True)
     entity_type = Column(String)
-    package = Column(JSON)
+    package = Column(JSONB)

magla/db/directory.py

@@ -1,5 +1,5 @@
 from sqlalchemy import Column, ForeignKey, Integer, String
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import relationship
 
 from ..db.orm import MaglaORM
@@ -15,8 +15,8 @@ class Directory(MaglaORM._Base):
     user_id = Column(Integer, ForeignKey("users.id"))
     label = Column(String)
     path = Column(String)
-    tree = Column(JSON)
-    bookmarks = Column(JSON)
+    tree = Column(JSONB)
+    bookmarks = Column(JSONB)
 
     machine = relationship("Machine", uselist=False, back_populates="directories")
     user = relationship("User", uselist=False, back_populates="directories")

magla/db/episode.py

@@ -1,5 +1,5 @@
 from sqlalchemy import Column, ForeignKey, Integer, String
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import relationship
 
 from ..db.orm import MaglaORM
@@ -14,7 +14,7 @@ class Episode(MaglaORM._Base):
     project_id = Column(Integer, ForeignKey("projects.id"))
     directory_id = Column(Integer, ForeignKey("directories.id"))
     name = Column(String)
-    otio = Column(JSON)
+    otio = Column(JSONB)
 
     project = relationship("Project", uselist=False, back_populates="episodes")
     sequences = relationship("Sequence", back_populates="episode")

magla/db/facility.py

@@ -1,5 +1,5 @@
 from sqlalchemy import Column, Integer, String
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import relationship
 
 from ..db.orm import MaglaORM
@@ -12,6 +12,6 @@ class Facility(MaglaORM._Base):
 
     id = Column(Integer, primary_key=True)
     name = Column(String)
-    settings = Column(JSON)
+    settings = Column(JSONB)
 
     machines = relationship("Machine", back_populates="facility")

magla/db/file_type.py

@@ -1,5 +1,5 @@
 from sqlalchemy import Column, Integer, String
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import relationship
 
 from ..db.orm import MaglaORM
@@ -12,7 +12,7 @@ class FileType(MaglaORM._Base):
 
     id = Column(Integer, primary_key=True)
     name = Column(String(20))
-    extensions = Column(JSON)
+    extensions = Column(JSONB)
     description = Column(String(100))
 
     tool_versions = relationship("ToolVersion", back_populates="file_types")

magla/db/orm.py

@@ -25,7 +25,7 @@ class MaglaORM(object):
     """
     # `postgres` connection string variables
     CONFIG = {
-        "dialect": "sqlite",
+        "dialect": "postgres",
         "username": os.getenv("MAGLA_DB_USERNAME"),
         "password": os.getenv("MAGLA_DB_PASSWORD"),
         "hostname": os.getenv("MAGLA_DB_HOSTNAME"),
@@ -76,6 +76,7 @@ def _drop_all_tables(cls):
     @classmethod
     def _construct_session(cls, *args, **kwargs):
         """Construct session-factory."""
+        print("session constructed")
         # TODO: include test coverage for constructing sessions with args/kwargs
         cls._Session = cls.sessionmaker(*args, **kwargs)
 

magla/db/project.py

@@ -1,5 +1,5 @@
 from sqlalchemy import Column, ForeignKey, Integer, String
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import relationship
 
 from ..db.orm import MaglaORM
@@ -14,7 +14,7 @@ class Project(MaglaORM._Base):
     directory_id = Column(Integer, ForeignKey("directories.id"))
     timeline_id = Column(Integer, ForeignKey("timelines.id"))
     name = Column(String)
-    settings = Column(JSON)
+    settings = Column(JSONB)
 
     timeline = relationship("Timeline")
     settings_2d = relationship("Settings2D", uselist=False, back_populates="project")

magla/db/sequence.py

@@ -1,5 +1,5 @@
 from sqlalchemy import Column, ForeignKey, Integer, String
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import relationship
 
 from ..db.orm import MaglaORM
@@ -15,7 +15,7 @@ class Sequence(MaglaORM._Base):
     episode_id = Column(Integer, ForeignKey("episodes.id"))
     directory_id = Column(Integer, ForeignKey("directories.id"))
     name = Column(String)
-    otio = Column(JSON)
+    otio = Column(JSONB)
 
     project = relationship("Project", uselist=False, back_populates="sequences")
     episode = relationship("Episode", uselist=False, back_populates="sequences")

magla/db/shot.py

@@ -1,6 +1,6 @@
 from magla.db import episode, sequence
 from sqlalchemy import Column, ForeignKey, Integer, String
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import relationship
 
 from ..db.orm import MaglaORM
@@ -17,7 +17,7 @@ class Shot(MaglaORM._Base):
     episode_id = Column(Integer, ForeignKey("episodes.id"))
     sequence_id = Column(Integer, ForeignKey("sequences.id"))
     name = Column(String)
-    otio = Column(JSON)
+    otio = Column(JSONB)
     track_index = Column(Integer)
     start_frame_in_parent = Column(Integer)
 

magla/db/shot_version.py

@@ -1,5 +1,5 @@
 from sqlalchemy import Column, ForeignKey, Integer
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import relationship
 
 from ..db.orm import MaglaORM
@@ -14,7 +14,7 @@ class ShotVersion(MaglaORM._Base):
     shot_id = Column(Integer, ForeignKey("shots.id"))
     directory_id = Column(Integer, ForeignKey("directories.id"))
     num = Column(Integer)
-    otio = Column(JSON)
+    otio = Column(JSONB)
 
     assignment = relationship("Assignment", uselist=False, back_populates="shot_version")
     shot = relationship("Shot", uselist=False, back_populates="versions")

magla/db/timeline.py

@@ -1,5 +1,5 @@
 from sqlalchemy import Column, ForeignKey, Integer, String
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import relationship
 
 from ..db.orm import MaglaORM
@@ -13,6 +13,6 @@ class Timeline(MaglaORM._Base):
     id = Column(Integer, primary_key=True)
     user_id = Column(Integer, ForeignKey("users.id"))
     label = Column(String)
-    otio = Column(JSON)
+    otio = Column(JSONB)
 
     user = relationship("User", uselist=False, back_populates="timelines")

magla/db/tool.py

@@ -1,5 +1,5 @@
 from sqlalchemy import Column, Integer, String
-from sqlalchemy.dialects.postgresql import JSON
+from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import relationship
 
 from ..db.orm import MaglaORM