Proposed improvements

[layertwo]

@v1tal3, I have a few ideas on some features I'd like to go implement, but I want to run them by you first so we can be on the same page and not accidentally end up implementing similar ideas.

Here's a list of features/items I'd like to work on:

1. Rework user authentication to user Flask-Login and save in database
   a. Future work to include LDAP support
   b. This really helps with implementing route authorization rather than the frontend templating trickery that's going on
   c. Also, API integrations (point 3), role-based access control, etc.
2. Integrate a credentials store for server-side SSH
   a. I believe you may already have done something like this by dropping credentials into Redis, but I'd rather have those stored in the database and not a memory cache.
3. Rework templates and Javascript to pull from an API
   a. This would reduce the work on server template rendering and allow others to build automation/tooling around this
   b. I'm thinking this would include drastically reducing the existing templates and having most frontend rendered by some Javascript framework
4. Asynchronous server-side polling of hosts
   a. Right now when pulling up a new host, the page can take a while to generate as everything in sourced on the fly from SSH and then rendered on the templates.
   b. I'm thinking something like the server SSHs into a host and indexes it in its database every 15 minutes. When an update is made, make that change in the database, and have the server run that command asynchronously.
   b. Points 2 and 3 alleviate the transition to this.
5. Ability to upload JSON templates for new devices
   a. My thought here is that we generalize common functions most appliances would have (uptime, show interfaces, VLANs, etc.) and use system specific commands based on the type of appliance. I believe you're already doing something like this, but having the ability for end user to craft their own JSON templates allows for faster implementation of new hardware.
6. Refactor and reduce the number of functions
   a. It could be my lack of full understanding of what all is written, but I'm seeing a lot of this function call that, which calls that, which calls that, etc. I'd like to refactor and reduce making it easier to follow for other developers.
7. Python3 support
   a. We've talked about this before, and it wouldn't be too terribly difficult to do. I've played around with it before, but I run into some issues and am having trouble following the full code path to resolve the issue.

These are some high-level items I'd like to implement. Let me know your thoughts!

Thanks,
Lucas

[matt852]

Wow, these are excellent ideas. Let me respond to each individually:

1. Good idea
   a. Yes. Same with RADIUS/TACACS support. See credentialing section below
   b. Agreed
   c. I like the API (such as with flask-restless) and RBAC. Those were two things I had planned on working on sometime down the road. Excellent ideas.

2. See credentials section below

3. Sounds good, but it sounds like it would require the API idea in 1c. to be implemented first?

4. This is a great idea, and something I toyed with when first starting out on this project. I also played with (as an example) pulling the interface config every X minutes, then when a user logs in, it could show them if there were any changes made, when, etc.

   However I went with providing real-time insights into the switch, at the sacrifice of waiting to connect each time you accessed a switch. My justification was it was a similar waiting period when logging into the switch via Putty.

   I think this is still a great idea and I'd love to implement it. My ideal solution would be what you mentioned as the default behavior, with a button or something on the "viewspecifichost.html" page to force a full refresh of the data currently on the screen. Thoughts?

5. Agreed, another excellent idea.

6. You're right on this. A lot of it was code ported from existing scripts I had, and I never really did a full cleanup of Python code/functions. I've done partial ones here and there, but there's a few things that really need to be done soon, before I add any more new features. See the section at the bottom for my current rough roadmap.

7. 100% agree (see rough roadmap below). If you tell me where you're stuck at, or why, I'll take a look at your branch and try and help out. Thanks for starting this already.

Credentials:
I've had this issue for a while now, and I'm very much open to better ways of handling it. The problem I have with using SSH into Cisco networking devices is I have to be able to pass the username and password to the device via SSH, since they don't support SSH keys. Therefore, I have to be able to store the password and be able to decrypt it when accessing the device.

This is why I do not store the credentials any longer than necessary in the app, and I store them server side. I know session variables are stored in an insecure client-side cookie, so I don't like that. I'm not sure how Flask-Login implements this, but I'm open to it if it works.

In short, I'm open to any better way of authenticating to devices securely. I just haven't figured out a better one that works for what I'm trying to do yet.

Current Rough Roadmap:
I'm planning on keeping 1.3.1 the last release with new features (excluding bug fixes) for now, and really working on a few foundational items that really need to be done, some you've already mentioned.

• Python 3 support
• Better handling of credentials
• General code cleanup (such as redundant functions)
• Cleaning up the general Flask framework code (specifically views.py; it's a giant mess!)
• Better handling of simultaneous connection requests (currently one page loads at a time, and if you queue up four connections, it takes a long time for the 4th to load)
• CI Testing

For CI testing, the app is growing pretty big and I'm having a much harder time testing everything before pushing a release. I still haven't figured out an easy way to automate tests on this, since it requires a live Cisco switch/router to log into. It's something I've been discussing, but I'm open to ideas on this.

Overall, thanks for these great ideas. I was about to start working on converting to Python 3, but since you've already started that, I'll help in your branch, and start working on general code/function cleanup, and tackling the views.py file.

[layertwo]

Thanks for the response!

1c. Is actually easy to implement and I've already done this on a branch I do have.
3. Yes, I actually view a lot of this as one giant project as I can't have Javascript scrape an API for table rendering on interfaces without having that in the database. That likely won't be in the database until the server can asynchronously poll the device.
4. Yes! My thought here and explained above to is that each host should be indexed on a set interval (I'm going to recommend a default of 15 minutes), but allow the user to initiate a "right now/update host" option.
Architecturally, I want to create a model for interfaces in the database for which the data can be indexed, trended, and polled via API.

Re: credentials
This is certainly a hard piece to figure out. Flask-Login has the option to hash a password. This can be handled for user accounts created in the app. For situations where the app needs to poll a host, this won't work. Flask-Login is hashing the password and then comparing what the user puts in against its hash. The case where the app needs to poll a host, it could save those credentials in the database encrypted, but they need to be decrypted on the way out. This isn't very strong realistically. I'll do some digging into alternatives, but the first thing that comes to mind would be to Base64 the password (we all know how strong that is).

Re: roadmap
Some of the branches I do have a bit experimental have bit-rotted a bit. Personally, I would hold off on porting over to Python3 until existing functions are cleaned up. Some of the work, in my opinion, for cleanup happens when we start moving to local logins and credentials stored in the database. This removes most of the reliance on Redis (we'll bring it back for async). The roadmap I think would probably be best would be this:

• Local logins
• Credential storage
• API (flask-restless, couple lines of code)
• Javascript polling API
• Cleanup, refactor and build unit tests for existing functions (already happening above)
• Asynchronous
• Python 3
• Future: templating

Thoughts on this? I'm going to go rebase the branches I do have for logins and credential storage and can provide a PR for that shortly.