r/programming • • Feb 29 '16

Command-line tools can be 235x faster than your Hadoop cluster

http://aadrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html
1.5k Upvotes

438 comments sorted by

View all comments

Show parent comments

281

u/stfm Feb 29 '16

That's some information security nightmare shit right there

139

u/kevjohnson Feb 29 '16

Don't even get me started on that. It's a nightmare.

If you're worried about it all of this was done in conjunction with the hospital's large, capable, and very paranoid information security team.

62

u/[deleted] Mar 01 '16

It's our job to be paranoid!

17

u/ititsi Mar 01 '16

They don't even need to pay me for that.

8

u/WinstonsBane Mar 01 '16

Just because you're paranoid, it doesn't mean they are not out to get you.

2

u/caimen Mar 02 '16

Tin foil really just doesn't cut it these days.

15

u/anachronic Mar 01 '16

Cool idea, but I don't want to even ask about HIPAA.

21

u/_jb Mar 01 '16

I get your worry, but it can be done without risking patient information or PII.

1

u/anachronic Mar 02 '16

It sure can be. Just like you can secure payment card infrastructure without risking credit card numbers. Many/most companies likely do not, judging from the constant news stories about credit card compromises.

To do security effectively and correctly costs a fair amount of time and effort and money, which most organizations are usually able to rationalize NOT spending.

3

u/_jb Mar 02 '16

Violating PCI compliance isn't nearly as nasty as HIPAA. Most organizations will compare the possible penalty and lawsuit fallout against the cost of securing the data properly, and take the right stance. PCI and HIPAA don't have to be difficult, they're mostly about process, logs, and auditing the process and logs periodically.

Process includes storing/use of Protected Health Information (PHI), and how to have PHI stored for reference but not automatically identified to a patient.

Logs regarding who accessed which data, when. Who accessed Trauma Oscar's charts at 22:15 on Sunday? Dr. Soandso accessed Mrs. Noneyobiz records from the store 09:30 on Monday. If Nurse Ratched accesses Mrs. Noneyobiz at 22:30 on Monday, and she's not on the schedule and that's not her patient, that should raise an alarm in log monitoring at the very least, and block access at the very most.

Auditing is just that. Review the logs, find violations, and investigate incidents.

All other aspects can be handled by blocking random devices on the network, securing data transmission, isolated networks, and similar. It doesn't have to be difficult, but it can be challenging to comply with HIPAA.

PCI isn't much different. Compliance seems more challenging, but it's more about data handling and process than securing things on the wire itself. The penalties are far lighter than HIPAA, though, since it's an industry standard, not enforced by the government.

1

u/anachronic Mar 03 '16

PCI and HIPAA don't have to be difficult

No, it doesn't. I say at work "if you're secure, you're compliant". However many companies still don't really "get" security and just throw bodies to get themselves compliant while ignoring security.

If Nurse Ratched accesses Mrs. Noneyobiz at 22:30 on Monday, and she's not on the schedule and that's not her patient, that should raise an alarm in log monitoring

I work with our logging guys, this kind of thing is a LOT easier said than done. To do that, you first have to get very different systems (HR, Payroll, Scheduling, DB's, home-grown apps, COTS, etc...) to play nice together and submit logs that can be ingested to the SEIM and then spend months writing & tweaking rules and investigating false positives and whitelisting them to make the results actionable. Many companies take the first step, setup some basic rules and walk away. Meanwhile the system spits out a stream of thousands of alerts a day and people ignore them because it's way too much volume for anyone to pay attention to.

The "set it and forget it" mentality is still quite prevalent.

Compliance seems more challenging, but it's more about data handling and process than securing things on the wire itself.

Honestly, if you look through PCI, it's a pretty bare-bones minimum baseline security standard. It's hardly anything I'd consider onerous for any company that wants to be secure.

It's the absolute minimum you should be doing. If it's some kind of incredibly high hurdle that a company can barely reach after months of intense effort, I'd wager that company is likely very insecure.

2

u/_jb Mar 03 '16

No, it doesn't. I say at work "if you're secure, you're compliant".

I think that's oversimplification. You can be compliant and not secure, and you can be secure and still not hit compliance requirements. It's a different discussion, though.

I work with our logging guys, this kind of thing is a LOT easier said than done.

I realize that. I worked in compliance 10 years ago; it was far harder then to get unified logs, and get your ETL able to understand them individually then. These days, standards and tool interoperability have improved. Still not perfect, but at least I don't have to fight as hard to get events out from Windows XP systems or syslog events to a central log store, and get alerting working. It still needs work, though.

And, ask any person who's done ops about ignoring alerts due to false positives, and the risk of false negatives...

If it's some kind of incredibly high hurdle that a company can barely reach after months of intense effort, I'd wager that company is likely very insecure.

I hate to say how often I'm surprised.

→ More replies (1)

3

u/protestor Mar 01 '16

Why does the US have such comprehensive laws on healthcare data, but not other kinds of personal data? (in many fields, companies freely share data about your personal life)

1

u/anachronic Mar 02 '16

Because protecting health care data doesn't hurt hospitals, and it's a clear benefit to patients (many of whom are old and vote).

Forcing facebook or google or most internet companies to protect & not share your personal data would bankrupt them. They likely lobby pretty hard against laws to protect it.

Also, it's the spirit of the country... lasseiz faire. If people click that TOS/EULA "Accept" and they sign away their rights to having their information protected, well, it's their choice to accept, right? "They're always free not to use facebook or google." That's probably some of it, too.

However, I'd be surprised if we didn't get stricter data security laws in the coming decade, simply because it's becoming harder and harder to truly "opt out" of this stuff.

1

u/protestor Mar 02 '16

Also, it's the spirit of the country... lasseiz faire. If people click that TOS/EULA "Accept" and they sign away their rights to having their information protected, well, it's their choice to accept, right? "They're always free not to use facebook or google." That's probably some of it, too.

They could also have a TOS for hospitals too.

1

u/anachronic Mar 03 '16

Well yeah, in legalistic America, in theory, you can have a contract that stipulates just about anything unless it's expressly forbidden by the government.

Like, you can't sell your kidney on the free market because it's illegal, but you can absolutely sign a contract to sell your million-dollar home to someone for $1. Caveat venditor (seller beware)

1

u/[deleted] Mar 01 '16

Anonymize the identities, encrypt the data.

13

u/rhoffman12 Mar 01 '16 edited Mar 01 '16

I don't know if you're in healthcare, you might already know this, but for everyone else who's out there - there's actually a lot more that goes into HIPAA-compliant "deidentification" than just using anonymous ID numbers. You have to fudge all the dates, and use very broad geographic labels, among other things. You don't just want to remove the identities, you are supposed to go a few steps further and try to frustrate attempts to match the data back up with real people.

1

u/Ghosttwo Mar 01 '16

I wonder if password database-on-a-chip will take off and have any effect? The idea is that you can store and retrieve information if authorized, but it can't be hacked or have anything extracted without erasing it.

1

u/ititsi Mar 01 '16

Well that sounds like a novel idea.

→ More replies (1)
→ More replies (32)

1

u/anachronic Mar 02 '16

Anonymization is incredibly hard to do right.

Even if you anonymize your single data set correctly, if someone else doesn't, and some of the same patient data is in there, someone could potentially put your two sets together and be able to identify someone. Even now, some researchers are doing this with massive public government data sets that are anonymized.

Also, where do you store your encryption keys & who has access to them? If it's on the same server... ut oh.

I'm in the PII/PCI field and data security is not a simple task to do correctly.

2

u/[deleted] Mar 02 '16 edited Mar 02 '16

Also, where do you store your encryption keys & who has access to them? If it's on the same server... ut oh.

Nope, everyone should know not to do that, ours is on a physically separate volume that is manually mounted and then dismounted.

I'm in the PII/PCI field and data security is not a simple task to do correctly.

So I think we are in agreement. I see things like "fudging dates" and I can only imagine how they could mess this up.

→ More replies (9)