Dear Tim,
I had a conversation with a friend of mine today about the current status of Kalmany. Part of it was because I finished the work I needed to do in the API gateway to get the API responding and integration, and we had a quick discussion on that part of the system.
As per my philosophy, ‘if it works for what I need, that’s what I care about’, especially when it comes into some of the long-term business considerations that can occur when dealing with public APIs, I built the thing and I talked about some of the considerations I had.
One of the hurdles I had to deal with was CORS that conflicted with my development, as I created a DEV API and yet it couldn’t be accessed by my Kalmany local deployment. Eventually I figured it out (I had to check a box to enable CORS and set the ‘Access-Control-Allow-Origin‘ header to ‘*’) and I could finally link my local development without it breaking. Now, this won’t remain the case – obviously I’ll need that header to just be my domain, but I’ve not yet a domain nor a certificate. Until I set those up, the CORS will be a bit too open for public consumption, but as nobody knows about Kalmany, I’m not worried.
The other consideration that comes is that, being a React frontend, by API endpoints are blatantly visible. There’s a way around this of course, AWS’s Cloudfront (which I already need to add a HTTPS certificate to my site) can act as a proxy and allow me to limit the access, but there’s not really a way to prevent people seeing that.
But this is one of those ‘do I really care?’ situations because my API is entirely GETs. It’s all information retrieval. Protecting my API boils down to preventing people DDOSing as there’s no way to affect the data in my database using the API. If someone access my API directly, that sounds like bad practise, but in the end they’ll be retrieving the exact same data as my frontend displays! Except it’s in JSON and looks fairly unpretty. I could be incredibly sensitive to who accesses the API, but I really don’t mind if someone were to call the endpoints directly – the only way they can break the system is by DDOS. Or injection.
Now injection could be an issue. However, I don’t deal with user input at all. The only way someone can manipulate injection into my platform is if they accessed my API and decide to mess around with the path parameters. But the simplest solution here is the best – I expect very specific inputs. For example, my /constituency/ endpoint allows for the specification of the constituency_id, which then is used for the /constituency/<constituency_id>/demography and /constituency/<constituency_id>/election endpoints. This could easily be manipulated, but then again it can’t. I have 24 constituencies, no more, no less. Their IDs are basically static. I can easily limit the API to run the right SQL statement as long as the constituency_id is between 1 and 24 and error for anything else.
Because I’ve got a very cut-in-stone design for the operations of my website, I have the potential for incredibly specific control on interactions, which saves me from a collective ‘what-if’ that are present when dealing with cyber security. The actions that can be taken are incredibly small, so I don’t have to worry about a ‘blacklist’ that doesn’t cover everything, I can easily implement a ‘whitelist’ so to speak and only approve the actions I’ve designed for.
Things will get complicated when I introduce authentication, but for now my worries should remain limited.
Now DDOS methods I can prevent in API Gateway itself, it already has options to limit traffic. If I put it behind Cloudfront, it does it double. This worry will just be to make sure someone doesn’t make a request to /assemblyfeed/ (which at the moment can respond with a 1000 results) a 1000 times and overload my database and/or my API. I can rate limit of course, which is beneficial as I’m trying to deal with keeping my AWS platform from costing me anything all, but then I also have to consider dealing with an API that returns 1000 results. That’s a lot of rows. I could paginate and complicate my code to cycle through pages. I could paginate my React code so the AssemblyFeed has pages rather than a continuous block (but this I don’t want to do. I like it a big scrollable div). Or I could code the Kalmany backend to just NOT CREATE 1000 MESSAGES. There’s going to be a way I can reduce this number. Every member of an assembly (24) has to speak for each member’s policy (24) so instantly I’m going to get 576 messages. Each might have a success or fail attribute so now I’m at 1152. Each one needs rendering.
But there’s going to be a way to reduce this. The messages are all kept in the database, but really this isn’t necessary, there’s a lot of string repetition. I could restructure the code to instead store the data in a much simpler way, with a success column, an action column, and so on. Then when it renders the data in React, it renders the string based off these values. I could then reduce the 1000 to 500 just about. Unfortunately, I think that’s as far as I can go so in either case it looks like I might need pagination in the API.
The final thing is domains, and CloudFront seems to be where I need to launch some more investigation. In essence I just want a HTTPS frontend, and I want my domains to be recognisable and not some amazon garglemesh. It might end up being overkill but there’s no way around it for S3. My API Gateway may be saved from it, unless I decide to proxy it, but like I say earlier I’m not bothered if people access my API directly. If they want the fake data, take it.
But this will get complicated when authentication is on the line. Can’t start letting people access other people’s data, fake or not. I’ve been leaving that to last as it will start to really affect my systems when I’ve got to rely on asynchronous data being input into the system. Luckily it can be contained and I will want to ensure it’s contained so that a person’s individual data won’t be part of Kalmany’s citizen data. And in the end, if Kalmany ends up in the public eye, the citizen data will be immensely unimportant. Those poor people. Their democracy will be overtaken. I hope their happy with having their fates determined by zealous gods.
Anyway, I’ve heard reports that the Swiss police are looking for a Tom, so I don’t know how you’ve managed to give them the slip, but just make sure you protect the jade badger. You know the Austrian Mafia won’t take kindly to their prize possession being broken, even if you’re currently holding a fake. I just hope the real one does lead to treasure, like you theorise it does. If Captain Mauvebeard did exist (and I hope he does), you may be able to rebuild your father’s smoking lounge. Good luck!
Yours,
Stan
WoodenSocks 