Given the keys never leave the KMS hardware encryption module, are you at all concerned that all your data will be destroyed if you lose access to KMS for any reason? That's what has always given me pause when I consider KMS. Or do KMS users just take on faith that AWS will always be there?
Why do you like their auto-rotation? The keys that are rotated out are not never disabled, so I don't really understand the benefit. In what scenario would their auto-rotation improve security?
> Or do KMS users just take on faith that AWS will always be there?
I don't take on faith that they'll always be there, but I do believe that if for whatever (extremely highly unlikely) reason they did go away that they'd make it possible to get my keys, or give me enough notice so I could re-encrypt with other keys.
Face it, when running a business there is a ton of trust you have to put in 3rd parties (banks, insurers, your employees, the government, etc.) Yes, you should always evaluate the trustworthiness of 3rd parties, but AWS going away and deleting my keys is probably #6327 of things I worry about.
Can you comment on how compatible this is with asyncio-based applications? Looks like an interesting product and I'm going to try it out regardless, but it would be cool to get some clarity since I couldn't find anything in the docs besides this:
> Time (blocking call) profiler supports threads and gevent.
We haven't tested the whole agent with asyncio applications yet. I guess only CPU profiler was tested during development. We'll do and include it in the docs. For now, if you see any problems, please just open a ticket. Thanks!
Neither Django nor Flask is asynchronous/event loop based. It would neither make sense nor improve performance to run a Django/Flask app on top of uvloop. If you want to develop a web API using uvloop, you should look at aiohttp (http://aiohttp.readthedocs.io/en/stable/).
I'm not really sure what the point of this article is. If you're clever, you can represent any data as a bit array. And once you're there, counting bits or XORing them together is easy. But is that system easy to understand? Is the code easy to read for the rest of the developers who work on the project? Or for future hires? What happens if the data set becomes too large to fit in memory on a single machine? Storage is cheap and getting cheaper every day. The whole data set described in this problem (done the "inefficient" way) fits on a flash drive at today's capacities.
There are always tradeoffs when it comes to architecture design. Speed and storage space are only two of the many factors that require consideration.
I don't understand this argument. Is it really likely that a spaghetti nest of Python, json, Hadoop, and Elasticsearch is easier to understand and to maintain than maybe 100-200 lines of self-contained Go code with minimal (no?) outside dependencies?
Or that new hires are going to think bit counting and XOR are ninja CS voodoo, while everyone already knows how to Elasticsearch?
How about cost? What's the monthly all-in cost for each approach?
This is a nice satire. It points out why modern software can be faddy, inefficient, fragile, hard to maintain, and expensive to run, with no obvious benefits (including "easy to understand", which seems to be code for "I already know how to use all those dependencies and APIs, so let's assume new hires will too.")
Not at all. I can appreciate an elegant, problem-specific solution as much as anyone else. The point is that there are always tradeoffs.
What's the query interface like for the bit array? Does it even have one? Seems like you'd have to sit down and add more code whenever you wanted to know something new about the data. Would you write your own query layer? Would you eventually need a dedicated team to maintain it? You may say that is an extreme conclusion, but I've seen this very story play out multiple times in large dev organizations.
As amorphic pointed out, indexing the data with ES makes it easy to access even for non-technical users. Each layer of abstraction comes with its own costs – of course – but also its own benefits. Tradeoffs.
> What's the query interface like for the bit array?
I don't see any request for a query interface in the list of requirements.
The point of the packed-bit array is that you don't need any kind of query interface other than a few accessor macros. It's a memory-mapped array of 64-bit integers; you just index into the array and use a macro to mask out the bits you're interested in.
The "old-timer" solution - which actually solves the stated problem instead of a hypothetical future problem with lots of extra requirements - can be written in a few pages of C. It shouldn't take more than an hour, perhaps two (plus some time for testing).
If and only if the requirements change0 to something significantly more complex would something like a query interface make sense (YAGNI). The task as stated just isn't very hard.
I haven't used go, but it's easy in C. Setting up macros for all the bit manipulation is a very common technique. Usage would be trivial - just call a couple accessor macros. It's certainly easier than walking the parse tree of a JSON record. Using a simple bitmap would also skip the initial parsing step.
It's easier to use the bitmap. The "example usage" section below is very simple. Setting up ElasticSearch or fiddling with a JSON parser is more work (and a lot harder on CPU/RAM).
/* the "2nd array" */
uint64 *port_states;
#define IP_PORT_STATES(ipaddr) (port_states[ip])
#define PORT_22_INDEX 0
#define PORT_80_INDEX 1
// ...etc...
/* some of the accessor macros */
#define PORT_STATE_MASK (0x00000003)
#define PORT_STATE(ipaddr, port_index) \
(PORT_STATE_MASK & (IP_PORT_STATES(ipaddr) >> (3 * (port_index)))
#define PORT_CLOSED 0
#define PORT_OPEN 1
// ...etc...
/* example usage */
uint32 open_count = 0;
uint32 ip = 1;
do {
if (PORT_STATE(ip, PORT_22_INDEX) == PORT_OPEN) {
open_count++;
}
} while (ip < 0xffffffff)
> future hires
If they can't handle calling a couple macros over a uint64 array, I wouldn't recommend hiring them.
> too large to fit in memory
It's not necessarily in memory - the "old-timer" is using a memory mapped file.
Because the last time I wrote a bitmap like that was about a decade ago using a proprietary compiler for a Z80 clone that implemented a "C89-like"[1] language? Call it a bad habit from too many years working with embedded micros.
If I was using a modern(-ish) C, an enum might be more appropriate.
[1] When you only have 16k of RAM and a 256 byte stack, having "int x;" default to "static" storage instead of the stack is a feature.
Ill add that having your data in Elastic means that a (probably non-technical) user still can use it as part of their analysis without having to get dev involved.
Well, yeah. That's not the intended use case for Slack. People do use it for public groups organized around a shared interest, but it is intended for teams or companies.
Actually, the ruling stated that there was no evidence of Yelp behaving the way they were accused of. It went on to say that, even if they were, it would at worst constitute hard bargaining and not extortion.
I worked at Yelp for two years as a software engineer. I have seen everything there is to see: the code, the databases, etc. There is no extortion table. There is no extortion code. Businesses like to make this claim because it generates sympathy and attention. It's just not true.
Why would any of it show up in the code? As I recall, what Yelp is accused of is making fake bad reviews that go away if you give them money. That could easily be a manual process done by sales people.
The sales people don't have access to the content moderation systems. As far as it showing up in code, I was just making the point that there is no file containing something like
if business_has_paid_extortion_money:
show_good_reviews()
else:
show_bad_reviews()
... which is how people seem to think it works.
There is also no code to allow manual review ordering. They are sorted according by an algorithm with a variety of inputs including who you are friends with, who has Elite status, etc.
It could be because you posted a link to Jezebel, which is largely reactionary manufactured-outrage shlock. "Hey guys the courts totally said that Yelp is free to do whatever extortion they want, isn't that crazy?! Get your pitchforks out!" Their style is just not conducive to civil, well-reasoned discussion.
I love the design of Sunrise, but the slowness of the app and network refresh infuriates me. Been using the Google Calendar app this morning, and love the instantaneous sync.
Sunrise was relegated to the fourth screen on my iPhone immediately.
Loving the calendar+tasks hybrid view. It even resizes as you interact with either so you can fully make use of the available the screen estate. One can say it's "Touch-Responsive Design".
There’s also a feature you can enable that automatically rotates your key once a year. KMS is great!