Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feedback #1

Open
wants to merge 11 commits into
base: feedback
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Ignore credentials.json file
credentials.json

# Ignore timestamp.txt file
last_sync_timestamp.txt
46 changes: 46 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
[![Review Assignment Due Date](https://classroom.github.com/assets/deadline-readme-button-22041afd0340ce965d47ae6ef1cefeee28c7c493a6346c4f15d667ab976d596c.svg)](https://classroom.github.com/a/AHFn7Vbn)
# Superjoin Hiring Assignment

### Welcome to Superjoin's hiring assignment! 🚀
Expand Down Expand Up @@ -59,3 +60,48 @@ All the best ✨.

## Developer's Section
*Add your video here, and your approach to the problem (optional). Leave some comments for us here if you want, we will be reading this :)*

## Current Approach:

Google Sheets API:

I'm using the Google Sheets API to facilitate communication between Google Sheets and the Express.js backend, to query, update, and manage data in Google Sheets programmatically.

APP Scripts for Automation:

I'm using App scripts within Google Sheets to automate tasks such as ID generation and timestamp creation.
Timestamp Generation is used to record when a cell or row was last modified for accurate synchronization.
This is done to identify and update only the modified data, reducing the need to process the entire sheet each time.

## Initial Focus:

CRUD Operations:

Establish CRUD (Create, Read, Update, Delete) operations between Google Sheets and the database.
I will then establish similar operations from the database to the sheets using triggers.

Future Enhancements:

Pub/Sub Model:
Explore converting the synchronization process into a pub/sub (publish/subscribe) model if time permits.
Benefits:
Scalability: Handle high loads and real-time updates more efficiently.
Decoupling: Separate the data update process from the data synchronization process to improve system responsiveness and reliability.
Additional Considerations:

Error Handling:

Implement comprehensive error handling and logging in both the Express.js backend and Google Sheets scripts.
Ensure that any issues in synchronization are detected and resolved promptly.
Testing and Validation:

Conduct thorough testing to validate the synchronization logic and ensure data consistency.
Use sample datasets and edge cases to verify that CRUD operations and timestamp-based updates work as expected.

## MY PROGRESS:
I was able to complete all primary goals which were part of my inital focus. Additionally, I was also able to establish efficient modifications since the sheet or db is only updated using "Id", which removes the necessity of going through the whole db or sheet.

The biggest challenge I faced was to sync the sheet on updating the db. I considered using triggers for this, but the issue is that these triggers activate every time there's a change in the sheet, which creates a loop since the database updates also trigger the sync process. To tackle this issue, I came up with a solution to efficiently enable and disable these triggers, which seemed like the easiest choice with the time left.

VIDEO LINK:
https://drive.google.com/file/d/1l21aS6cnY_QrCQ5ZpGrHTVJ0iDCuoFXh/view?usp=sharing
18 changes: 18 additions & 0 deletions config/db.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
const mysql = require('mysql');

// Set up the connection to the database
const db = mysql.createConnection({
host: "localhost",
port: 3306,
user: "root",
password: "aditya11",
database: "superjoin",
});

// Connect to the database
db.connect(err => {
if (err) throw err;
console.log('Connected to the database.');
});

module.exports = db;
19 changes: 19 additions & 0 deletions config/googleSheets.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
const { google } = require('googleapis');
const path = require('path');

// Google Sheets API setup
const auth = new google.auth.GoogleAuth({
keyFile: path.join(__dirname, '..', 'credentials.json'),
scopes: "https://www.googleapis.com/auth/spreadsheets",
});

// Create client instance for auth
const getClient = async () => {
const client = await auth.getClient();
return client;
};

// Instance of Google Sheets API
const googleSheets = google.sheets({ version: "v4" });

module.exports = { googleSheets, getClient };
20 changes: 20 additions & 0 deletions index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
const express = require('express');
const { syncSheetToDb } = require('./services/syncService');
const { syncDbChangesToSheet } = require('./services/dbChangeListener');

const app = express();
app.use(express.json());

// Sync DB changes to Google Sheets every 10 seconds
setInterval(syncDbChangesToSheet, 8000); // Check for DB changes every 10 seconds

// Sync Google Sheets to DB every 10 seconds
setInterval(syncSheetToDb, 8000); // Check for changes in Google Sheets every 10 seconds

app.get("/", (req, res) => {
res.status(200).send('Periodic synchronization in progress.');
});

app.listen(3000, () => {
console.log('Server running on port 3000');
});
12 changes: 12 additions & 0 deletions node_modules/.bin/mime

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions node_modules/.bin/mime.cmd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions node_modules/.bin/mime.ps1

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions node_modules/.bin/nodemon

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions node_modules/.bin/nodemon.cmd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions node_modules/.bin/nodemon.ps1

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions node_modules/.bin/nodetouch

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions node_modules/.bin/nodetouch.cmd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions node_modules/.bin/nodetouch.ps1

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions node_modules/.bin/semver

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions node_modules/.bin/semver.cmd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions node_modules/.bin/semver.ps1

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions node_modules/.bin/uuid

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions node_modules/.bin/uuid.cmd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading