Dev Log
Dev log for this project
5/30/19 - This project is pretty a big hosh podge of a bunch of things I want to learn, much like the Microservices project was a big project on learning microservices, consul, gRPC and vault. In this project I'd like to learn:
Make an API in node with HTTP2 capabilities
User Authentication through Firebase
Error Handling Best Practices (with Sentry)
Handling Nested routes
Endpoint Schemas (with Joi)
Automated Testing (with Jest)
Docker + Node & NGINX Best Practices
Logging (with Winston)
GCP Tools (hopefully, but not entirely sure)
ACLs with user permissions
CI (with TravisCI) and more. . .
So it's safe to say that this is going to be a big project. Even bigger than the Microservices Project. I hope to start today, but depending on how things go I might have to start tomorrow. Regardless, I look forward to getting this project up and running!
5/30/19- //DEV// So with a bit of research, it's shown that Express does not currently support HTTP2 natively. I'll have to use something similar, like Fastify or Restify. Restify is taking my interest atm.
6/1/19- //DEV//
So far, I've gotten > how to make an API in node with HTTP2 capabilities completed. Now I need to implement Firebase as a user backend for this.
6/2/19- //DEV// Completed HTTP/2 integration and pushed edits. Now I'm looking to cleanup my routes and implement some security and QOL measures. I'm thinking along the lines of limits, order by, pagination, auth permissions, and CORS. I also decided that once I finish Automated Testing with Jest, I'm gonna try adding pen testing mitigation techniques and automated pen testing! I'm really looking forward to trying out pen testing for my API to see what I can fix about it! Overall, got a lot done today, even with stupid errors getting in the way haha.
6/3/19- //DEV// Simple sentry integration is complete, now to work on limits, order by, pagination, auth permissions, regex, and CORS.
6/4/19- //DEV//
Okay, so there's a lot that I'm not understanding at this moment, so I'm going to try to type this out and hope that it helps organize my thoughts. I have completed the > how to make an API in node with HTTP2 capabilities , and created a simple implementation of sentry. Once I add more to the API, I can start scaffolding out bigger error handling. So the first and third task are accounted for.
[Question about #2] The second task is a bit harder for me to try to wrap my brain around. I understand that in order to authenticate to Firebase Auth, you need to have some sort of client set up to actually request to your server for authentication. However, I am a bit confused on this. 1) Is the client (assuming you wrote it in node) equivalent to the browser, or can you open up the client from the browser?
[Concerns with #4, 5, 6, 8, 9] These seem pretty simple to set up, I just want to apply the use case correctly. I don't want to make it just for making it's sake. So these features are tabled for the time being, until I am able to fully scaffold out the use case I'm trying to demonstrate in this project.
[Concerns with #7] I believe that this is what I will work on until I answer my initial question about #2.
6/5/19- //DEV// Okay, its time to reassess what this project is going to encapsulate at what point. I think the following additions are a much better suited for my experience level and what I intend to learn:
Standard HTTP/2 Node API with 2 - 3 endpoints
TLS between the API and the underlying database
Order-by, Limit, and Pagination query options
Regex integration
NGINX Web Server Config
Rate Limiting on NGINX
Securely storing configuration specifics in Consul
Securely storing secrets in Vault
7/6/19 - //DEV// So, from last time a lot has changed! I was able to get #1, 2, 3, 4 and 5. I wasn't able to get 6, 7 and 8 for a few complications: 6 - I have yet to get this configured yet because I have to get it working on my machine and then pull over the configs. Hoping to do that sometime soon 7 & 8 - So, this is something that I learned recently, in order to Consul & Vault in a production-style secure way, you have to have some overarching "authority" that holds your secrets. And since I don't know kubernetes (yet), I don't have much of a choice other than just putting it off until I know kubernetes. Soooooo, what now? Well, right now I'm kind of between a lot of things.
Express & HTTP2 (and maybe HTTP3) Madness As I've talked about before, express doesn't natively support the HTTP2 module (coming in express 5.... whenever that is 🙄 ). A lot of what I've seen online from this point is to just use express + spdy for http2, which would be a great alternative! 😒would be...The issue with this is that spdy breaks for node versions >=11.1.0. So what I've had to do was use Restify as supports HTTP2 natively. Not what I wanted to do, but its pretty much all I had. And then I saw how express lets you do express.Router() and abstract your router out, which is great!... But in order to do that in spdy, you'd have to add another dependency... So right now, I'm stuck with either dialing back node to a much older version, or adding another dependency to my stack, which is something I'm trying to avoid because dependencies are just more things to leave security holes. Along with ALL of this, HTTP3 is in the works too, so that'll probably be another 3 to 4 years of trying to get that adopted too 💀
Dockerization This isn't really something that I'm thinking about. I'm going to do it, it's just a matter of when will I make the time to wrap the NGINX parts nicely like they should be.
Continuous Integration + Testing I didn't forget, I swear! Haha! My plan right now is to set up very basic testing in postman, export that testing suite to a file that will be stored in the repo, and continuously build off of that. From there, I will have TravisCI handle setting up those tests on every push. After I finish that and get things cleaned up, I plan to start testing with Jest on the utilities and services folders. This is because I believe that my postman tests will be the better part of my end to end tests, while Jest will handle all of the fine-grain testing.
Last updated