When Security Audits attempt to make your server move vulnerable.

So we host a few servers that have to be scanned for PCI compliance, if a server isn’t passing PCI compliance for some reason we hear about it. It’s usually because they’ve added a new client, setup them on a new IP and not telling us so we can run our firewall setup. It gets scanned and fails miserably because the shared services on the server (Email, Control Panel) etc are exposed when we custom build the firewall for each customer to only allow connections from certain locations.

Then we also have some customers that have some CentOS 5 servers, CentOS 5’s built in version of openSSL only allows SSLv3 and TLSv1. SSLv3 is disabled for security reasons as it should be. But then we get a a security auditor that starts screaming about a BEAST a browser vulnerability that has been mitigated in clients as it should be by using 1/n-1 split. This security auditor was using 3 year old information and failed our customer telling them to instead use:

SSLCipherSuite RC4+RSA:!EXPORT:!LOW

in apache.

This sets up the customer for failure because RC4 has been known broken for a year now. And a scan from Qualys SSL analyzer gives that a fat F. So I told them to reply with this:

=======================================

BEAST stands for Browser Exploit Against SSL/TLS Attack, essentially if someone can get between your customers browser, and your website and listen in on traffic they can attack the browser and get cookies from the browser. This attack has been known since the Early 2000’s and was updated with TLSv1 in 2011 it’s impractical and besides lab proof of concepts hasn’t been seen in the wild, as most of the chain of client exploits required for even the proof of concept to work in a lab environment have been patched by their vendors.

BEAST can be mitigated server-side (by using RC4, a broken, and weak cipher where attacks are just going to get better and easier) or client-side (1/n-1 split, which has been implemented in every major browser for quite some time).

At this time it is better to use strong ciphers and leave TLSv1 enabled (Technically allowing a vulnerability with BEAST) then to disable TLSv1 (making your website inaccessible to ~25% of the browsers in use), or enable RC4 (Making your server vulnerable to the growing number of attacks against a weak cipher that has been known broken for over a year).

BEAST is not considered a relevant attack any longer, and RC4 is widely considered by the security community to not be a workaround worth using:

1.) https://luxsci.com/blog/is-ssltls-really-broken-by-the-beast-attack-what-is-the-real-story-what-should-i-do.html
2.) https://threatpost.com/not-so-fast-on-beast-attack-mitigations/102308
3.) https://community.qualys.com/blogs/securitylabs/2013/03/19/rc4-in-tls-is-broken-now-what
4.) https://www.isecpartners.com/media/106031/ssl_attacks_survey.pdf (See section 2)

=========================================

Apparently that did the trick. But with the PCI DSS 3.1 having just came out and with it saying TLSv1 having to be disabled by June 2016 we’re going to see a mad rush of folks needing to move to CentOS 6 or 7 from their previous cushy CentOS 5 havens. Which isn’t a bad thing as CentOS 5 reaches end of life in March of 2017, it’s a bit dated now.

And sometimes you are just wrong.

Making mistakes as a sysadmin happens. It happens more often then we like. The difference between a good sysadmin and a bad one is that the good sysadmin will own up to their mistakes, point out how it happened, and come up with methods in which to avoid those same mistakes down the road and share what they learned with their team; where as a bad sysadmin will hide their mistake and hope no one calls them out on it.

My Drill Sergeant in basic training taught me that if I’m going to fuck up, that I need to fuck up so damn good that it makes those around me wonder if they were the ones wrong. If they call right face and you turn left you keep facing left until corrected. (Private Snuffy and Private Snuffy only. about. face.)

Given: there are times when that sort of thinking is entirely not compatible with reality, or that’s what the Highway Patrolmen told me when I was the only one going the right way on a one way street. Fuck those other 400 drivers.

It’s best when you can learn from others mistakes, it means you don’t have to personally make the same mistakes as another member of your team, which in turn makes the team work better. That takes trust though. Trust, that while you might get picked on for making the mistake, that the team will pull together and try and fix the problem instead of shunning you for your honesty. I’ve worked in both kinds of shops. My very first IT related job I thought my name was “What the fuck Chuck”. Looking back the “Seniors” were what I would call entry level now, and made fun of folks to elevate their own perceived standing in the company. But I still learned from them somewhat even if sometimes what I learned was 100% how not to do something.

If you are in an environment where this happens and you are in some form of leadership the easiest way to resolve that sort of toxic environment is to fess up to your own mistakes to your team. If not, move on, life is to short to spend in a toxic environment.

Charles~

So you want to be a technician / IT specialist.

Do you like being a front end to Google? Because that’s how you become a front end to Google.

No seriously, as a child I used to think “Man, adults have this life thing figured out.” Nope, no we’re all just kinda winging it. Working in IT: if you know something, good; if you can figure stuff out, even better. Whatever you may know, one day it could change when a half drunk coder fixes the bug that makes that golden fact-that-you-know irrelevant with a barely coherent commit on github. Knowing how to research or find the information is just as, if not more useful.

Then you have deep specialist vs generalists. Having the guy that only knows regular expression frontwards and backwards and can parse log files after writing a 3472 character regular expression and passes the results off to a tech that can solve it is entertaining. Having a generalist that has to use grep, awk, sort, and awk again to get the same results and takes 3 times as long because she has to look up the man(ual) pages or –help listing of each command but then is able to turn around and knows how to use those results to solve the problem is much more valuable.

How do you get there? Tinker with shit, and learn how to sift through mountains of bullshit that Google throws at you. It’s not really Google’s fault, most people with any sort of technical know-how suffer from an advanced case of dunning-krueger syndrome and would rather tell someone to defrag their hard drive to resolve a virus problem and look like they are helping someone than to admit “yeah I don’t know what the fuck I’m doing”, much less admit if they are wrong. Help forums are fool of people like this. Note: fool was a typo, but damn it I’m leaving it in there. Also expect to find help postings where no solution is given but the original poster goes “never mind figured it out” and never says how they fixed it(FUCK these guys), or posts go unanswered.

Learn a scripting language, bash, perl, or even fuck I’ve written irc bots in PHP. Figuring out how to automate stuff is a valuable contribution. There does come a time though when spending hours and hours automating something that will only work until the next major release of some tool will break it, that you are in a bad time spent/time saved trade off. So learning that point and never stepping over it (unless it’s just for fun at that point), is very valuable. Your boss pays you to do shit, basically trading your expertise to work on tasks that take X amount of time. If they are getting a shitty automation that they can only use once to save 2 hours but took you an 8 hour day, it’s not worth it.

Learn how to take technical problems and make them into analogies. You would be surprised how easy it is to get people to understand DNS when you tell them it’s basically a phone book for the interwebs. How easy it is to get people to understand the difference between hard drive space and RAM when you say “a hard drive is like your filing cabinet, and ram is like the top of your desk, you can work on papers while they are still in the filing cabinet but it’s much slower then if you took it out and wrote on your desk. The problem is your desk doesn’t have that much space (like RAM) but it’s faster, where as the filing cabinet is slower but can store more.”

Users are fucking dumb. This is a fun saying, in some cases it’s actually true. But for the vast majority of people you are going to deal with their expertise just lies in other areas. You just happen to be good in an area they aren’t. So…it creates a job opportunity. Never demonize the user for not knowing what you know, instead find ways to make money off this, because they are doing the same to you when they do your taxes, prescribe your medicine, deliver the cases of beer you drink to numb the pain of how excruciatingly dumb sales people are as you pray to whatever deity will listen that it’s not caused by something that is contagious.

Also consider the above, if you are in a situation where you are in a support role you have to remember that you are seeing peoples problems, you aren’t seeing their triumphs, it’s like the reverse of facebook where all your high school mates are fucking rockstars and have perfect children except for Becky who airs out all her drama at the top of the hour every hour she is awake and the only reason you don’t unfriend her is because it’s like watching a train wreck in slow motion. Don’t worry though, Becky gets back together with the guy that cheated on her with an entire roller derby squad.

If you can handle all that, IT is for you. If not…

Charles~

Once again, I’m back.

It’s been about 6 years since I had this blog open, have a trip down memory lane if you like. My how time has gone by. So, what the hell happened?

Around 2008 I stopped working for the business that was creating my dire need for catharsis by lambasting every idiot that came through the store, I went to college and started my own business. Without the antagonistic co-workers pissing off customers, and the ability to “fire” customers, I no longer felt the need to write about them. Well that and my old employer / now competitor was aware of the blog and might have used it against me.

So 4 years of running my own business I got fed up with it and started looking employment ANYWHERE else and after flying across the states for in person interviews I wound up choosing where I’m currently at. Now, I prefer to keep it all secret of course, we must protect the innocent guilty, and anonymize this whole thing up.

The place I’m at is rather relaxing, but it does allow me many opportunities to learn, and in fact since I’ve been here I can say I’ve learned quite a bit. So while I won’t be griping about customers near as much as I used to, I will be sharing what I’ve learned. And hopefully, I’ll make it fun too while I’m at it.

Charles~