Welcome to Online Tech!

I Can Hack You

Thu, 2 Jul 2009 14:12:21 -0500

«Ï» çÅñ H@Çk ¥°Ü· What the hackers do, and how to keep them from doing it
The Making of a Hacker
Picture, if you will, a parasite that calls itself an "immunity tester." Our immunity tester travels from host to host, "testing" to see if it can feed off that particular host. When the feeding is good, our parasite tosses a few chunks of meat in front of the sharks. The feeding frenzy begins.
You'll know the feeding frenzy when it happens. You've been hit by the password traders. Unless you have frenzy protection, your bandwidth use will go through the roof. Someone needs to pay that bandwidth bill - but it won't be the sharks or the parasites.
In spite of the feeding frenzy, our parasite is a responsible parasite. He's careful to not destroy or otherwise damage the host. Sure, he starts the feeding frenzy, but the frenzy itself is not his concern. He'll add one or two passwords to your members area rather than two hundred. Two hundred might get noticed.
If he's cracked hundreds of your passwords, he'll only post a few at a time for the sharks. You'll focus on the few, without realizing that hundreds are known. As our parasite trickles out the "fresh" passes, he remains a hero for continually doing such great work.
Almost every hacker board calls itself the top resource for "security testing." They are "educational" in nature, and not for profit. If you're a webmaster, you can ask the board owner to remove all references to your site from their board - and they will.
Hackers divide their work into two basic categories, cracking and exploiting. Cracking is the process of decrypting passwords so as to gain access to the members area of a paysite. Exploiting is the process of doing stuff you weren't supposed to be able to do, so as to gain a copy of the password file for cracking purposes, or gaining access to other interesting information on the paysite's server.
The sharks, by the way, must move quickly. That's the reason for the feeding frenzy. There are so many sharks looking for fresh passes, that the passes last mere hours or even minutes. Shark traffic hits with a sudden heavy surge.
As a freeloader, how do you get away from the other sharks killing your passes? Your two choices are time, or money. For ten dollars a month, or twenty five, you can become a "special" member with access to longer-lasting passes. Yes, the not-for-profit board owners are selling stolen access to your bandwidth. It's a tough sell, though. The freeloaders are freeloaders.
Your other option is to become a hacker yourself. Create your own passes! You can keep them to yourself, in which case they may last forever, or you can post them. Posting them makes you a valued contributor to that board. Since you're now a valued contributor, you'll also have access to that "special" area. You can now steal porn to your heart's content.
The best part is... it's easy, with no prior experience required. Exploiting requires you to use your brain, but beginning cracking does not. If you're a clueless freeloader who wants to get all that porn for free, cracking is the place to start. Let's begin.
John the Ripper
Step One. I stopped by the Openwall Project (www.openwall.com), home of Solar Designer's legendary John the Ripper password cracker. I installed it and grabbed a password file (with permission) from one of my customers' paysites. I ran John the Ripper with its default settings like this:
./john passwordfile
How simple is that! I had dozens of passwords cracked within about 20 minutes. I did that my first time. I had no special word list, no specialized "cracking" rules, no benefit of other crackers' experience. I grabbed the rest of this customer's password files, and played with John the Ripper's various "modes" as I read through Solar Designer's help files.
By time I read through the help files and several beginners' tutorials available online, John the Ripper had spewed out several hundred cracked passwords. I knew it was good, but I was quite astounded by how quickly John the Ripper can "kill" that many passwords.
Step Two. I took those results, and several other lists I had laying around, and added everything to John the Ripper's wordlist. If "apple" is in the wordlist, for example, John the Ripper might try such things as apple, Apple, and apple1 through apple99. By feeding your list of usernames and your list of known passwords into the wordlist, you create a snowball effect. John the Ripper can make more educated guesses based on what's been found already. As you go through cycle after cycle of cracking and feedback, John the Ripper becomes that much more powerful.
I cheated a bit... I seeded the list. I grabbed a bunch of cracked password listings off the hackers' boards. I added each of those usernames and passwords to my wordlist. Since I already had the listings saved on my hard drive, I only needed about ten minutes to add a hundred thousand words to my list. (I then realized I'd inserted a lot of still-encrypted passwords, and took another ten minutes to clean the list.)
Step Three. I grabbed another password file, without permission. Once you're allowed into the "elite" areas of the hacker boards, you'll find the secret keywords for hundreds, probably thousands, of paysites. Depending on the "exploit" listed, you'll be able to add your own password to the site, harvest the live password file, or even look around the entire server. I pulled the password file.
John the Ripper cracked over a thousand passwords from that file within two hours! My total time spent, counting from when I first visited the Openwall Project to download John the Ripper, was less than four hours.
Imagine, if you will, someone who gets really good at this. John the Ripper can run on anyone's desktop PC 24 hours a day 7 days a week. Just feed it new password files occasionally, and post your results so the freeloaders can have "fresh" passes daily. It's so simple that kids can do it - and I strongly suspect that they do. Since it really is that simple, you can see why the "hacked password" problem is as large as it is.
A Side Note
I personally feel that the current ease of paysite hacking presents a serious legal issue. We are not exercising real diligence in keeping the material inaccessible to minors. Perhaps calling it a legal issue is putting it too strongly - for now. Yet the fact remains that fourteen year olds can crack passwords as easily as eighteen year olds.
Regardless of how they got there, minors can get in all too easily. In my opinion, that is a problem. The problem is easily solved - but we must choose to bother to do so. All we need for a short-term solution is (1) an appropriate password-choosing policy, and (2) properly-written billing scripts. For the moment, that really is all we need. (The better solution is to improve our authentication procedures. Human nature - and the Law of Software Inertia - guarantees that won't happen until we're forced to do so.)
As we continue "the making of a hacker," please bear in mind how easily minors can do this. Remember that legally speaking, it's our obligation to keep them out, rather than to expect minors to refrain from coming in. Remember that we actually can choose to keep them out.
Advanced Cracking
The next step would be to apply your skill and experience to the specific password file at hand. If you know all passwords are eight random digits, for example, you can search accordingly. John the Ripper has its own programming language wherein you can tell it what approaches to take.
Suppose you have managed to crack a few passwords, and discover that when the username is firstname.lastname, the password is first initial followed by last name followed by 1-5 random digits. You can crack the rest of the file almost instantly! Tell John the Ripper to keep the first letter, drop everything up to the dot, drop the dot and keep everything following the dot.
With a bit of experience, you can make this example even simpler. Many paysites use standard unix-type passwords, called DES encryption. Only the first eight characters of the password are encrypted! So, to use the example above, if the person's last name is seven characters or more, you know the password. No guessing is needed.
Do you see why this is so? The password (according to our assumed rules) is first initial followed by last name followed by digits. But... only the first eight characters are used. So, if the name is seven letters or longer, all the leftover characters (including all the digits) are ignored. So far as the paysite is concerned, the password is that person's initial followed by the first seven characters of their last name.
In the same way, if the person's last name is six characters, you need only try adding a single digit. That gives you a mere ten possibilities to try. Even if the last name is a single character, you only have a hundred thousand combinations to try. Since John the Ripper can run millions of trials per second, the worst possible case will still have you seeing dozens of passwords cracked per second.
Given enough examples, a good cracker can figure out the rules. Assigning passwords that are random ten-digit numbers would seem to make them tough to crack - until you remember that only the first eight digits count. Sure, a hundred million trials is a lot. But... it's possible. And... it's easier than it might appear.
Create a list for yourself of, say, one million eight-digit numbers. You now have 1% of all possible combinations. If your list is truly random, and if your list of passwords to crack is also truly random, you should successfully crack about 1% of those passwords. In theory you'll "kill" about one password per minute, which isn't all that bad. If your results do stay around 1%, you're right on track. However, if your results do not generally meet 1% success, you need to change your assumptions. You can perhaps find a better pattern, and crack a much higher percentage!
Generally, however, people get to choose their own passwords. There are no rules. Does this make cracking more difficult? Quite the opposite! People tend to choose passwords that are easy to remember. They also tend to use the same password (or a minor variation thereof) for several different sites.
Once you've gained the experience of cracking thousands or even millions of passwords, you'll have a pretty good idea of how people tend to make their choices. The huge wordlists (sometimes running to several Gig) allow even more passwords to be guessed outright. You can find other passwords by applying various rules.
For example, someone might think "office" is a good easy to remember password. But, to make it tricky, he changes it to "0ff1ce". The password consists of mixed letters and digits, with the digits not adjacent. That sounds good - except that so many people change "o" to the digit zero, "i" to the digit one, "e" to three. John the Ripper can follow the same rules.
Brute Forcing
Brute Force "testing" is basically a sparring match between the hacker's software, and your server's software. You now have a nice pool of username/password combinations, thanks to your John the Ripper adventure. What are the chances of that same username/password working for other sites? The chances are pretty good, it seems! Some silly combinations such as test/test and admin/admin get you into an amazing number of sites.
Brute Forcing, then, is the process of running down your list, trying each one to see if you get into the site. If you check your server logs, you'll probably see hundreds or thousands of failed attempts, often with usernames tried in alphabetic order. If you watch carefully, you may even see some successful attempts!
Brute forcing is so common that there's not much point in trying to block it. Or is there? Let's look at the sparring match more closely, so you can see what I mean. Take the example of a typical member surfing on in to your password protected members area. You're using standard "Basic" .htaccess authentication with DES encrypted passwords. Let's assume your members area is at /members/index.html.
(This example assumes you're running the Apache server software. However, the server and encryption details aren't the important point here. We're focusing on how "Brute Forcing" works. The general idea applies to form-based logins, session cookies, or pretty much anything else.)
When your member goes to the members area, his (or her) browser pops up a little box asking for the username and password. But... what happens under the covers? Your server logs will show you.
When your member first asks for /members/index.html, the server responds with code 401 (Authorization Required). Your member's browser pops the window asking for the login. The browser encrypts the username and password, and then reissues the request for /members/index.html. This time, however, the username and password get passed along as part of the request.
If the username and password are correct... but wait. How do you know if this is the correct username and password? You don't! What I'm getting at is this: Your site's password file might contain a thousand username/password combinations. If you supplied any combination on that list, you get in. If you were assigned username A, but correctly used username B, you get in.
In theory, if you know all of those thousand passwords, you can get in a thousand different ways. The server doesn't care. If you have a valid username/password combination, that's all the proof you need. The server does not care if the password was stolen or cracked. We make no attempt to tie that password to a specific person or specific IP number.
Typical paysite validation is actually pretty weak - and that's what makes brute forcing viable. Poorly-chosen passwords limited to eight characters are gloriously crackable - thus we have a large pool of likely passwords to try. The server doesn't care in the least if you're the rightful owner of that password. If you make a lucky guess, you're in.
Let's suppose you're running off a cable modem, and can try several passwords per second around the clock. That's a quarter million tries per day. With experience, and a good pool, you'll get in.
Or will you. Don't you think somebody will eventually notice? Probably not! Not until something major happens, like the server's error logs filling up a filesystem, or the bandwidth bill suddenly tripling.
Eventually, though, people will notice. If you're running off a cable modem, your static IP number shows in the server logs. The first ten million entries might get missed, but eventually you will get noticed. One change in the server or router configuration, and you're gone. Forever. You'll also receive a nasty note saying your cable modem account has been canceled.
Proxies
Fortunately, the Internet itself comes to your rescue. Rather than losing your cable account for being stupid, you can work through a proxy. Proxies have two main purposes (other than keeping the hackers in business).
The first purpose is in giving you a channel through a firewall. If you're on a secure corporate network, your computer might not have direct access to the rest of the Internet. What's important is the other way around: The rest of the Internet doesn't have access to you. Assuming you need some means of communication with the outside world, you communicate by way of a proxy. Roughly speaking, a proxy is to your browser what an email gateway is to your email client.
The second reason to have a proxy, is as a tremendously huge browser cache. AOL clients, for example, browse the world via the AOL proxies. You may have noticed that when you update a web page, AOL users may still see the older version for a day or more. That's because the page has been cached in the AOL proxy.
Part of a proxy's job is to be invisible. That is, it's designed to do its thing while staying out of the way. A proxy's function is to make requests on your behalf. It feeds things directly to your browser so that you never know the difference.
However, in the hacker world, proxies serve a truly critical function. They cover your tracks! When you surf into a site, that site's server log records your IP number. However, when you surf via a proxy, it's the proxy's IP number that gets logged. You're now much safer from those annoying letters saying your account has been canceled, or that you may face criminal prosecution.
Now you can do a quarter million Brute Force attempts a day without the server knowing who you are. If you're using one proxy, that one proxy can be blocked. So, the Brute Forcing programs allow you to rotate between, say, fifty proxies. You can specify the number of attempts per minute, when to pause for a bit, and so on. With its easy graphical interface, this tool is quite literally child's play. (Well, perhaps not child, but very probably early and mid teen.)
Do you remember I said that Brute Forcing is basically a sparring match? The standard Apache software makes no provision for dealing with Brute Force attacks. However, some sites do. They detect the attack in progress, and block things as best they can.
But... what do you block, when the attack is based on many different IP numbers and many different usernames? Each browser signature is different. As I understand it, servers will block each IP number temporarily. Use enough proxies, with enough time delay before reusing the proxy, and you may continue the attack.
Another thing the server can do, is lie to the attacker. The Brute Force attacker is looking for successful logins. Therefore, to foil the attack, the server can report everything as successful. The attacker, then, has gained no information from the attack.
That's why I call it a sparring match. The Brute Forcer's trying to fine tune the pacing of the attack, and the server's trying to identify what traffic is spurious while still keeping the paysite open to the legitimate members.
Are the best server admins going to tell you, or tell me, how they handle the attacks? Not a chance! Are the best Brute Forcers going to tell you every last trick of their trade? No. The best I can do is assure you that the battle continues, even though I'm not privy to the details.
Exploiting
Exploiting fits into two stages:
* Finding the hole (called scanning for exploits)
* Using the hole (called exploiting)
Scanning is easy. Pick a paysite and run through a list of URLs which might be interesting. You can download your own scanning program for free. You can do the same with other peoples' URL lists. You're supposed to then shorten your list to include only URLs that you personally know how to exploit. If you've noticed a bunch of weird off-the-wall URLs in your server logs from time to time, you've seen people scanning your site for exploits. You can safely ignore the scanning - unless they find something.
What they found, with the information necessary for its use, is called an exploit. Exploiters post lists of working exploits on the hackers' boards, the same as crackers post lists of working passes, as a means of sharing information. At the same time as the sharks are using the passes, the other crackers are adding those passes to their John the Ripper wordlist.
Several billing company scripts require a secret keyword - and nothing else. If you have that keyword, you can add members or display the entire password file. If that script name is "whatever.cgi" (it isn't!), exploiters know to check a paysite for /whatever.cgi, /cgi-bin/whatever.cgi, and so on.
Some exploiters check the join page to see if that billing company is being used or not. Some don't bother, since checking the usual places takes a second or less, whereas actually "eyeballing" the join page takes longer.
Let's suppose you have (1) found that whatever.cgi is there. You then found (2) the secret keyword, and (3) verified that the script is indeed exploitable (i.e., usable for your purposes). You then (4) harvested your own copy of the password file.
What's next? You have several options. (Remember, my purpose here is to explain what hackers do, so you can better understand how to keep them from doing it.) You can:
* Keep the information to yourself. You can crack the password file and post the results for the sharks to enjoy. You may choose to limit your posting to the "special" area. Or, if you have enough cracked, you might split your results between the "special" and regular areas.
* Keep the exploit to yourself, but post the contents of the password file for others to crack and use. You're a hero for having found it, and you're not giving away your secrets by telling how you found it. You can use that trick to find other files on other servers without other exploiters stepping on your toes.
* Post the exploit, not bothering to personally harvest the password file. You're a hero for posting the exploit, and you don't get any heat for getting caught. The other exploiters, in effect, become the sharks.
You might even pop in and enjoy the site. Most exploiters, though, seem to be too busy feeding the sharks.
Spoofs and Back Doors
Many sites use referrer-based validation. You get to visit the protected area if and only if you came from an authorized location. Since you needed a password to get to that authorized location, we assume everything is proper.
The most common referrer-based validation is with AVS sites, and with paysite plug-ins. The surfer enters his password on the AVS's server, and then proceeds from the AVS server to the AVS site's members area. Since we know the surfer came from the AVS server, we know he's authorized.
Third-party paysite plug-ins such as live video feeds often work the same way. So long as the member came from an authorized members area, he's allowed inside the video feed area.
There's an exploitable weakness in this system. We don't actually know where the surfer came from. It's the surfer's own browser which announces the referring URL. The server does not verify that information. All the hacker needs to do is supply the appropriate referring url, and he's in. He's spoofing his credentials.
Some paysites have "hidden" URLs. If you know where it is, you must be authorized! Once the back door is found, it will be posted, and suddenly your site becomes extremely popular. This technique works quite well for AVS sites which don't have referrer-based (.htaccess) protection.
Keeping the Hackers Out
You really have three problems: The sharks, the crackers, and the exploiters.
First, you need to keep the sharks out. Once a live password has been posted for your site, the feeding frenzy kills your server, and you'll be stuck with the bandwidth bill. So, given that your passwords have already been cracked, you must protect yourself from the freeloaders trying to get in. Your financial survival depends on it. (Several companies offer excellent shark-protection services.)
A good password-choosing policy will keep the crackers from feeding the sharks. It's that simple! The master crackers themselves confirm that they depend on people (including billing companies) using poorly-chosen passwords. When the passwords become uncrackable, they must either find another way in, or intercept them as plain text. (We'll discuss a possible password-choosing policy below.)
The exploiters are a very different situation. The exploiters are what we mean by "hackers" in the traditional sense of the word. All software has weaknesses, and those weaknesses can be found.
However, we really only care about paysite hacking. (We'll assume you have a good security-conscious server admin. You'll pay more, and it will be so worth it.) Here's the key: Nearly every exploit posted on the paysite hackers' boards is based on a weakness in a billing company script. Once you get the billing scripts cleaned up, you have quite probably thrown the exploiters off your server!
To be sure, hackers are hackers. They'll find a way back in. The paysite security procedures we use today are simply not adequate. Today's desktop PCs are more powerful than the codebreaking supercomputers of a decade ago. Desperately horny underage freeloaders have plenty of motivation - and time and tools - to find a way in. Still, if passwords become uncrackable, and billing scripts become non-exploitable, we've eliminated nearly every single thing (except spoofs and back doors) posted on the paysite hackers' boards today.
As you clean up your billing scripts, you need to remember that if you're sharing that server with someone else, you need to get the other guy's scripts cleaned up as well. When an exploiter breaks into the server through a billing script, he'll check the entire server for anything else of interest. Think about it: Can you really afford to share a server with someone who doesn't get hurt as badly if you get burned? You might pay more for a dedicated (rather than shared) server, and you'll be glad you did.
Password-Choosing Policy
Choose good passwords, and the crackers will have a rough time of it. What counts as a good password? If it's one that John the Ripper can't crack, it's good enough. If John the Ripper can crack it, it isn't. Fortunately Solar Designer himself (who wrote John the Ripper) tells us how to make sufficiently good choices. On his website http://www.openwall.com/crypt/ he explicitly places this information in the public domain so that you can improve your security.
First, we need the concept of a "character class." The character classes are digits, lower-case letters, upper-case letters, and other characters.
You must also bear in mind the limitations of your current encryption scheme. For example is upper-case considered different from lower-case? For unix DES encryption (standard .htaccess password protection), upper- and lower-case are different, e.g., "Fred" is a different password from "fred". How many characters are encrypted? DES only uses the first 8, so "Bullwinkle", "Bullwink", and "Bullwinkle&Rocky!" are the same password.
Generally speaking, if only the first eight characters of the password count, you need to include characters from all four character classes. A seven-character password must draw from all four character classes; an eight-character password can get by with three character classes. No password containing six or fewer characters is acceptable.
Since the username is in plain text, you need to watch for any portions of the username which are carried into the password, possibly as a reversed string and/or with i changed to 1, o to 0, e to 3.
Exploitable Scripts
I am not going to tell you how to exploit a billing script. Not here in print, anyway! However, you can pretty safely assume that unless you've done something special, your billing script can be exploited.
Generally speaking, paysite-related exploits fall into two categories: inadequate validation of authorization to use; and breaking into the unix shell with unexpected input data. Sometimes files are visible that shouldn't be, simply due to an improper server configuration. Billing companies seem to forget this possibility, and leave plaintext secret keywords laying around in those files.
Inadequate Validation
Sometimes it's trivial: Check /cgi-bin/ for the script, and it's there. Try it without any secret keyword or any other authorization, and it works. You're in! Some servers allow you to read .htaccess, the password file, or even the server log files. Check the usual places, and use what you find.
Sometimes it's a bit better, meaning more difficult for the hacker. You need a secret keyword to use the script. But once you find the keyword, you're in. There is no further validation.
So far as I can tell, many billing scripts were written by people that had no idea that hackers exist. If you hide the script, it must be safe. On the contrary, hackers have their own copies of the billing scripts, and they're able to read the code and recognize the weaknesses.
How can you be sure a billing script is secure? You probably need to take the same approach as the crypto people do. No crypto system is considered secure until it's been made public. Only when the public has examined it and proven unable to break it, is the algorithm deemed to be adequate.
Security based on secrecy is a false security. When you remember that minors can hack porn sites as well as anyone else, I would suggest false security is worse than no security at all. Assuming we actually care about our minors, we can't afford to keep kidding ourselves about paysite security.
Sure, billing scripts are proprietary. Nobody knows what they look like except the billing company, the webmaster, the server admin, the hackers, and google. Until they're made public, they'll probably remain as exploitable as they are today. But they're proprietary, and thus unlikely to become open to public comment.
Meanwhile, you can tighten up the validation, or have a third party do it for you. As I write this, the scripts which require a valid IP number (that is, it must match one from an approval list) are giving the exploiters fits. They haven't yet figured out how to forge an IP number - unless they can find a proxy within the script's allowed IP number range.
Breaking Into the Shell
Every script is different. The "best" scripts for exploiting are the ones that send user-supplied data into a system call without adequate validation. For example, one script expects the username and password as input data. If, instead, you supply a semicolon followed by a string of unix shell commands, you can find everything that server has to offer, including the secret keyword for the other company's billing script.
Just a Thought
The owners of the paysite hackers' boards complain bitterly about site rippers. Too many freeloading sharks are trying to harvest too many "free passes," and the boards are stealing from each other so as to have more passes to offer their own sharks. Thus the hackers' boards themselves would seem to be vulnerable to bandwidth concerns. Remove their paying customer base, or raise their bandwith expense, and they'll have the same financial pressure the rest of us feel thanks to their efforts.
Most sharks don't bother to hide where they came from. Your server logs will tell you where your site's passwords have been posted. While I won't presume to offer legal advice - I'm not qualified - I suspect you can plausibly argue to the board's upstream provider that you have been financially damaged by the information contained on that board. You can perhaps make the same claim with each shark's ISP.
Some of the hackers' boards do appear to be based in the USA. Some boards seem to be sheltered by that server's webspace provider. However, you may be able to find the provider further upstream less willing to condone the illegal hacking activity.
Just a thought.

How to Hack a Paysite

Thu, 2 Jul 2009 13:21:46 -0500

• How To Hack a Paysite What the Good Guys Need to Know
• One billing company did come to me and say they had changed their code as a result of reading my article. Another billing company changed their code as well, after rather thoroughly showing their president the mess they were in.
• If you're plagued by hackers and password traders, it's probably your own billing company who is letting them in. Is that news to you? It's probably news to your billing company as well!
• Your billing company could make hacking and password trading a thing of the past. Right now, though, there's no incentive. If you're plagued by hackers, that's supposedly your problem. You, not the billing company, are paying that bandwidth bill for the site rippers. You are perhaps even paying for third party password protection. As things stand now, you can't live without it!
• To understand the problem, I need to teach you how to hack a paysite. You'll understand what your billing companies are up against. And you'll see that with a remarkably small effort you can make your hacking problem a thing of the past.
• Getting the Attitude
• One particular password traders' board was running a fundraising drive. They were looking for donations from the members to pay for their own bandwidth expense. The board owner explained:
• Most of you have been receiving free passes since you joined and we have saved you $1000s of dollars. We dont think that a Donation is too much to ask for what you receive in return.
• The "free passes," of course, are the hacked passwords to your paysite. Most responses were whiners saying they were unemployed, but would kick in $5 or $10 when they were able. Yes, these are the people sending your bandwidth bill through the roof with their site rippers and traded passwords.
• One response, though, shows precisely what we're dealing with. Please take the time to read it in its entirety:
• Well I will just start by saying that I think that the owners should be compensated for the bandwidth. I first donated $25 when it was first asked by cueball, this is before gold passes were around and have continued every month to do so. Why, well, one thing its a great site with many great members, lots of stuff here besides passes. Second is the passes, I mean how many members would see even a fraction of these paysites without this site? The problem with most folks is they dont WANT to pay anything, they want everything FREE. Things just dont work that way. If this was my site I would certainely limit the number of those who arent paying so that I could reduce my bandwidth. Second, I would charge everyone a nominal fee, perhaps $10 a month to help offset the costs. Folks, the owners of this site owe you nothing free, so why don't you pony up. What the hell is $25 bucks a month? You can afford a puter, and ISP, and who knows what else. Time to pay the piper. My thanks to cueball and all the others that keep this site running and being such a wonderful site to be a member of.
• That's the attitude you're up against. The typical password trader doesn't even know he's doing anything wrong. The passes are there, so use them. If you can post a few "passes" yourself, you're a hero for being so clever and for sharing. By the way, the board's own USA-based hosting company is collecting the donations!
• Basic Paysite Hacking Technique
• Fifteen years ago, Robert Morris Jr. took down the Internet. This was pretty clever at the time, since it was the first such incident. The funny thing is, today's paysite hackers continue to use the same three techniques. Many of today's hackers - teenagers and college kids devoid of any sense of ethics - are too young to remember.
• Here are the three techniques. Neither you nor the hacker kiddies need to remember the details. The automated tools are waiting for you to use:
• A scanner. You've seen police radio scanners, right? It's a device which checks various frequencies, locking onto anything it finds interesting. Check your server logs, and you'll find lots of people looking for files and scripts you don't even have. They're scanning for "interesting" items, and will take note of whatever's found. (In the Morris Worm's case, the scanner was looking for more computers to infect.)
• A known vulnerability. This is called "an exploit" by the hackers. (Morris knew of vulnerabilities in the unix utilities fingerd and sendmail.) Nearly every exploit posted on the paysite hackers' boards is based on a billing company script.
• A password guesser. The passwords on your server are encrypted. So, they're safe, right? Think again! Morris showed how vulnerable they were fifteen years ago. Most billing companies seem to have forgotten this lesson!
• Let's Start Hacking
• So you want to hack a paysite. Where do you start? With a password file! If you can't find one on your own, visit one of the hackers' boards and you'll find 'em posted.
• Not so long ago - perhaps two years ago - a lot of paysites allowed their security files (.htaccess and .htpasswd) to be visible in a browser. If you knew where to look, there it was! Fortunately most server admins have closed up this hole.
• On the other hand... this trend seems to be reversing. More and more people have decided to save some money, and become their own server admins. There are more and more one-man hosting companies with great prices - but no security expertise. There are "web appliances" that will configure your server for you. By all means go the cheap route. The hackers will love you for it!
• Meanwhile, though, you have a password file. This is a list of all members of your paysite. The usernames are in plain text (john, jacob, jingle, heimer, and so on.) The passwords are encrypted (/Cphz8p6Emb3A, ooxdAVLkmR6/Y, auWXZ/088ALTQ, etc.). That makes them safe, right? Wrong!
• Here's how it works. Suppose you have the username:password listed like this:
• john:ooxdAVLkmR6/Y
• All we need to do is guess the password and try it. What Morris did fifteen years ago, is take the spellchecking dictionary right from the computer he was cracking, and run through the list of words as a guess. Often as not, he found a match! In the above case, try username john password john, and you're in.
• Here's the question. How good is your password policy? Are your users required to choose a password at least 8 characters long, consisting of upper case letters and lower case letters and at least one digit and at least one punctuation character? Better yet, do you assign random passwords such as 8p6Emb3A ?
• If you do not enforce a password policy, I can almost guarantee you that the majority of your users indulge in easy to guess passwords. They'll use a name, a birthdate, or the word Password just to be cute. If the hackers can find your password file, chances are they can guess most of your users' passwords! Today's cracking programs are extremely powerful, and wordlists run to 2 gig or more.
• A beginning cracker's rule of thumb is to crack 50% of the available passwords. And, generally speaking, they can! How will it affect your site if 50% of your paying members get locked out because of password trading activity? Fortunately, the password traders have "ethics" and "rules." They aren't supposed to post more than ten or twenty passwords to your site at a time. They go on to explain that otherwise the site owner might notice, and close up the hole!
• Your basic password problem is due to normal human nature. Most people will choose passwords that they can easily remember. The trouble is that if you can remember it, a cracker can guess it. The only solution is to require an extremely difficult to guess password. It does not matter whether you assign the password, or whether you allow the new member to make one up. What matters is whether a cracker can guess it within the next year or two.
• How and when does the password get chosen when your surfer signs up as a member? Normally they go to the secure join page, take care of their billing info, and choose a username and password. Here's where your password policy must be enforced! Who controls the policy at that point? Your secure transaction processor!
• I cannot emphasize this enough: If your billing company is allowing members to create easy-to-guess passwords, your billing company is responsible for your hacking problem. It really is that simple!
• To be fair, your billing company probably doesn't know this. That's because your billing company is run by nice people who don't know how to hack a paysite. But the technique of guessing a password has been around longer than your billing company has been online, and I assure you it is the primary beginners' technique in use today.
• I asked some of the master crackers on the Web, whether well chosen passwords are effectively uncrackable. Here are six typical answers. Read carefully, and you'll see the crackers themselves are telling us how to put them out of business:
• Hacker One. Nothing is uncrackable. It should take more time and you will need a little bit of luck, but if you use a good wordlist it could be done fast. But in most cases they choose not so clever passes. Also, extra characters like "=+*#$_" are really hard to crack.
• Hacker Two. 95% of people use very dumb and obvious passes! Also in 99% of the cases there is a logical sequence in them which is not that hard to figure out. I've written a little tutorial for... and it will almost kill everything if done right!
• If you do it my way you'll get everything up to 8 in length. There is another more difficult trick to go beyond that, but there is not much use for it. Usually usernames are longer but passes are in 99% of the cases shorter. This is to make for a quick entry (I guess...).
• Hacker Three. I agree with Hacker One. Nothing is uncrackable, but it may well be non brute-forcable, in that the only way to get access would be to view the passfile in plain text, because the passwords are so non-standard that no wordlist would be effective.
• The best example of this is a well known AVS. For example,
• user: ab98323432 pass: J54a7v6Q526eZ
• This password is clearly not guessable, or decryptable, nor brute-forcable by use of a wordlist, so the only way is to be able to view the password in its original plain text.
• Hacker Four. It seems to me that no one has pointed out the obvious fact that almost all adult/porn sites don't allow their customers to choose special characters in their usernames and passwords.
• There are some sites that let the members choose their own username, and then randomly generate a pass for them using uppercase, lowercase and digits in a wonderful hard undecryptable way (good example here is cdgirls and avs like AB and deluxepass).
• There are also sites that use 6-8 digits for both usernames and passes, but all-digit passes are reasonably easy to decrypt compared to a mix of uppercase, lowercase and digits. So the only way to get a chance of decrypting these kind of passes within a reasonable amount of time is to try and get clear text passes if these sites use MySQL or ORacle databases and gain access to these somehow.
• Hacker Five. The main reason people pick simple passwords (eg. apple1) is so that they can remember them.
• As Hacker Three mentioned there are a heap of sites (like AB) that generate their own passwords and send them to the user.
• This creates its own problems though. Let's say the guy whose normal pass is apple1 is named Dim Witt. Dim decides to join AB. He signs up and gets issued with DimWitt@hotmail.com:Aj4pIhDS4sMb
• What's the first thing he does? Write down the password because there is no way that he will remember it. He probably also keeps a couple of copies electronically. Now it may be harder for remote users to brute-force his password but not hard for anyone with direct access to his desk and or computer.
• One of the other problems with issuing Dim with a password is that he'll lose his scrap of paper and get kicked out of the site. This causes reverse workflows for the webmaster but they obviously think it's better than someone else using the pass.
• Hacker Six. The stuff that's being discussed here only counts when trying to guess a pass. If you have access to a server, it doesn't matter how "uncrackable" a pass is, because you can just see & save it. I've seen some servers that put their passfiles in directories like
• \web\pass\UG3264--_##UIvc-H87y3shghGkuGFGKS-\.htpasswd
• This is in no way to be found by brute force. But when you are there, what difference does it make?
• Someone else said it before here, but I totally agree... With new protections finding their way onto the web, bruteforcing as we know it will become a thing of the past. So Sploiting is the way to go...
• "Hacker Two" will crack passwords for you while you wait. Send him the password file and he'll have dozens or hundreds (depending on the size of the file) of "kills" for you in mere minutes. Modern cracking tools are that powerful, and the crackers' body of knowledge is that large.
• The Buck Stops Nowhere
• Let's step back a moment, and consider the situation. Your billing company should take responsibility for protecting your customers' billing information, and your server admin should take responsibility for protecting your paysite. After all, the billing info is on the billing company's server, and the paysite is on the admin's server.
• After carefully analyzing the situation, I strongly disagree. Your customers' credit card data is (usually) secure. Why is this so? Because that's what your billing company does. They have firewalls, authentication codes, secure logins, and surely keep a careful watch on outside probing. If they're being hacked, they know it.
• But who's keeping that close an eye on your members area? If someone got a copy of your password file, would you know it? If someone quietly added a couple of passwords, would you know that either? Nobody's watching! Your billing company doesn't consider it their responsibility. But if not, whose is it? Think about it: What are you paying them for?
• Exploiting the Billing Companies
• The basic problem is this: The billing companies are up against some serious technical difficulties when it comes to protecting your paysite. First, the server itself is outside their control. The server itself is your problem, and your server admin's problem. Second, there's a basic security flaw in how servers work.
• The first consideration is trivial: Go with a hosting company that knows what they're doing. You'll pay more, and it will be so worth it.
• The second consideration is how servers work. Your billing company has their own secure servers. When the transaction is complete, they need some means of updating your members area so that your new customer may surf on in.
• How do billing companies do this? By installing a set of CGI scripts on your server. When you first opened your paysite, the billing company probably came in, did their thing, did a test signup, and pronounced you ready for business. So far, so good. Let's call that CGI script the billing script.
• What does your billing script do? It updates your members area password file. This is how your billing company adds a new user, deletes an old user, changes a user password, and so on.
• Here's the problem. Generally speaking, your server makes no distinction between one CGI script and another. If the billing script has permission to update your password file, any other CGI script on your server can do so as well! Since your billing script is obviously able to read your password file, that means any other CGI script could also read your password file - if it were successfully asked to do so.
• If a hacker were to manage to get a CGI script installed on your server, obviously you're screwed. Your password file is wide open. And, as I explained above, your password file is gloriously crackable. Fortunately, getting a script inserted on someone else's server is rare, but it does happen. (Would your admin know it if it happened to you?)
• However, it is not hard to find an exploitable script on your server. An expoitable script is one that a hacker can use to get a peek at your password file. Remember, if one CGI script can read the password file, every CGI script can read that password file. That's the basic flaw in how unix/linux servers work.
• When it comes to exploiting CGI scripts, reputation doesn't mean anything. Matthew Wright's formmail.pl is one of the most notoriously exploitable - yet I have a $49.95 book on the shelf next to me, CGI/Perl Cookbook by Patchett and Wright. Matt's Script Archive is well known with high traffic.
• A lot of people out there use the formmail.pl feedback form. Somebody sticks in their email address, and the form data gets sent to the site owner. Let one user install such a script anywhere on the server, and every paysite on that server is now wide open. That's the kind of problem your billing company is up against! (I believe Matt has a more secure version of formmail.pl available these days, but that doesn't affect the existing installations.)
• Everything is readable
• So you want to hack a paysite. Do you remember where to start? With a copy of the password file. With a copy of that file - even though the passwords are encrypted - you can literally get cracking. But... how do you get a copy of that file? Isn't it protected from casual view?
• The basic rule of thumb is this: On a unix/linux server, you have to assume ANY file is readable. It probably can't be edited, but it can be read. That's how unix works.
• The unix folks learned years ago - Morris did the teaching - that the master password file is vulnerable. Since it was readable by anyone on the server, that was a problem. Unix itself changed over to a shadow password file, which was not so easily read. It's now quite rare to hear about a unix password file being found and cracked.
• Unfortunately, the billing companies don't have the luxury of hiding your password file. It must remain readable. What's really unfortunate, though, is that the billing companies - several of them, at least - failed to learn the Morris lesson. They completely forgot that any file on a server is readable! Where there's a will, there's a way... and that's an expensive lesson to forget.
• Can you imagine leaving a password laying around in a readable file, in plain text? If you can read the file, you have the password. Several billing companies code their master password right into the script, in plain text. Yes, really! If you can read the file, you have the password. If you have the password, you own the paysite. You can add, change, delete members at will.
• Did you catch that? Your members area passwords are encrypted. Most of them are crackable, but they are encrypted. However, several billing companies leave the billing script password on your server in plain text. The billing script encrypts the members passwords, but keeps its own password as plain text!
• As you can well imagine, these billing scripts are very popular with the hackers. When someone manages to find a copy of that plaintext password, they'll post it on the hackers' board so that everyone can use the script.
• Billing Exploits
• Is somebody exploiting your billing script? How would you know? Many billing scripts have no tracing, no audit trail, no other validation or protection. If you have the keyword, you're in. Your financial data is safe, but your paysite is wide open.
• Other scripts do a bit better with authentication, and do leave an audit trail of sorts. The irony is, that very audit trail is highly prized hacker food! The audit trail shows the passwords, and all too often they're crackable.
• If you visit the "elite" areas of the paysite hackers' boards, you'll see that the billing scripts are the most commonly published method of breaking into a server. If you're having a hacker problem, it's very likely that a billing company is your problem. The problem could be your billing company; or the problem could be someone else's billing script on the same server. (This is why you are far more vulnerable on a shared server. You have to worry about your own scripts and the other customers' scripts.)
• Have you noticed that I left out a step? To crack the passwords, you need to display a copy of the password file. You can trick a billing script into thinking you're the billing company, and add your own password. If you can do that, you don't need to crack someone else's password! But that doesn't get us a copy of the entire password file.
• The other thing you can do with a billing script, is break into the server itself. You can use the billing script to display files that you shouldn't be able to see. You can also display folder contents that you shouldn't see. That's how you find the things which are hidden, and how you find the secrets of other paysites on the same server.
• How, then, do you get a copy of the password file, so you can crack it? You use a billing script, somewhere on the server, to display the file. Some billing scripts come with online instructions for the hackers. It tells you how to display the password file, how to add your own username and password, and even how to delete all paysite members.
• Online help for hackers - remember that this is helping the hacker to hack your paysite - is nice, but far from the worst. Other billing scripts allow your hackers to run any unix command anywhere on the server. They can install their own stuff on your server, or scan your private areas for hidden webmasters' notes. It doesn't get much better than that!
• Exploitable Leftovers
• You as a paysite owner grow and change. You might change billing companies; you might move to a different server or ISP. You might open another paysite, and another, and another.
• As you grow, it's remarkably easy to copy unneeded scripts forward, and to leave unused scripts still laying around. You might not be using them, but the hackers still can! Those scripts are just as live, and as vulnerable, as they ever were - and chances are you'll never notice if they get used. Nor will those scripts ever receive security patches or updates.
• Solving Your Hacker Problem
• I've left out certain critical steps to successful paysite hacking. I've also failed to tell you exactly how to exploit the various CGI scripts. If you need proof, I can provide plenty. But the surest proof is your own server logs.
• Talk to your billing company. They're your friend - or should be. They have a stake in your success, and vice versa. You have two key issues to discuss.
• Members Area Password Policy
• Billing Script Security
• The difficulty, of course, is that these two areas are extremely sensitive. If your customers discover their password has been lifted, they'll also assume their credit card information is unsafe. This is quite untrue - but your customers don't know that! You know that billing information is treated far more carefully than is their paysite username and password. That difference is, in fact, probably the basis of your hacker problem.
• Even more difficult is discussing the security of your billing scripts. Every company considers them proprietary. Never mind the fact that you can find indexed copies in google, complete with master password. Nobody cares to admit to security holes. Especially when the admission carries the implication that credit card information might be no better cared for! We all know the credit cards are safe. We know the problem is that members area passwords are treated with less respect. But your members don't know that.
• The hackers themselves will tell you that if you limit yourself (and anyone else on your server) to difficult passwords and better-secured billing scripts, they will have a difficult time indeed. They may even go broke because they can no longer charge their members for the chance to steal your bandwidth.
• Additional Reading
1. Server Secrets. This article explains the unix/linux security hole and why it's necessary. If you ever struggled with permissions problems, this is the tutorial for you! (I have placed Server Secrets here as a separate article.)
2. A Tour of the Worm. The definitive article describing the full story behind the Internet Worm of 1988. It's actually interesting reading!
3. Murphy's Law. The original article is long since gone, “Murphy’s Law was Born Here,” perhaps because it has now been published as a book. It’s linked from the wiki page. The article is interesting reading and remarkably relevant: If something could possibly go wrong, you are responsible for assuming it will go wrong, and protecting yourself accordingly. (You must assume hackers will find a way to read files on your server. They will!)
4. Google. The number one piece of advice I see repeated on the hackers' forums is, Google is your friend. When you find a security hole, a google search will usually provide instructions on how to take advantage of it.
5. Google. If you're running mysite.com, for example, and your billing script is /cgi-bin/billing.cgi, try a google search for billing.cgi. You'll probably get complete instructions for exploiting the scripts, along with various vulnerable paysites.
6. Google again. For an eye opener, type username and password of mysite.com in the google search box. This little trick works darn near every time!

Server Secrets

Thu, 2 Jul 2009 12:09:53 -0500

Server Secrets
N.B. I wrote this six or seven years ago.
As your web business grows bigger and bigger, you will probably need to begin using other peoples' software. You might install a shopping cart, or use a gallery page builder... but sooner or later, you'll have to learn more about scripts and files.
Have you ever had a "permissions problem" with your unix/linux server? That's what I'm here to explain.
Well, no, that's not quite true. I'm here to explain several Server Secrets. The "permissions problem" is only one of those secrets. Unfortunately, this stuff is incredibly boring. We're not talking plain boring here... we're talking sadistically boring. But, if you survive the experience, you may never have to ask for help with your server again.
For our purposes, unix and linux mean the same thing. If you are on a unix-like server, this essay applies to you. FreeBSD, Red Hat, Slackware, Linux, Solaris, all refer to flavors of unix.
I am not here to teach you unix. I am here to help you survive unix as a webmaster.
File Ownership
Let's start with perhaps the trickiest concept of all - file ownership.
I'm sure you're too young to recall this, but back in the mid 20th century, there was something known as a typewriter. A typewriter typically had one owner - that is, any specific typewriter was only used by one person.
In the same way, your laptop or desktop computer is probably a single-user machine. You don't need to worry about working around or interfering with somebody else's files. You might worry about people snooping, but that's a different issue. Generally speaking, each person has his or her own personal computer.
There was, of course, a gap between everyone (or everyone's secretary) having a typewriter, and everyone having their own personal computer. We had computers, but they were shared. On a small to medium scale, we had departmental computers. Each person had their own account. Same computer, different accounts, just like your local bank. That's how we kept things separate.
That's when unix came to be. Unix evolved on medium-sized computers, in universities and research laboratories. Same computer, different accounts. Therefore, the idea of ownership became extremely important. Every file, every transaction is owned by somebody, and who that somebody is, makes a difference as to how that transaction is handled.
Again, think about how things happen at your local bank. Every dollar, every coin - and most certainly every ball point pen - is owned by somebody, and in every case it's completely clear who that somebody is. You have an account number; other people have an account number, and those numbers do not get mixed up. Keeping those account ownerships separate is a fundamental part of the system.
In the same way, different bank employees can do different things. At my bank, the lady at the information desk can handle non-cash transactions such as depositing my Google payout check. But only the teller can handle a wire transfer.
Unix is built around that same kind of environment. Even if you have a unix machine all to yourself, you still have to deal with the same concepts of file ownership. Perhaps you have a dedicated server; you are the only one on the whole server. But, the concept remains. You need to deal with file ownership.
Do you recall I mentioned that unix is based on departmental computing? I can think of no reason whatsoever for you to care about that fact. Even so, that fact does make a difference to you!
The files on my unix account are owned by me. That makes sense, right? The same concept applies to your server. When you upload your files, you specify your ftp user name. Your uploaded files are owned by that ftp user name. That is, your files are owned by you. When you deposit money into your own account at the bank, the money is owned by you.
So, on unix, there are a number of different user accounts. Each account has its own files - and thus different files are owned by different users.
However, on unix, a user is also part of a group. Back in the late 20th century, different departments wanted to keep their stuff away from the other departments, but share things among themselves. For example, in the following file list:
drwxr-xr-x 4 ewbarnard users 4096 Mar 30 19:24 . drwx------ 5 ewbarnard users 4096 Mar 30 19:20 .. drwx--x--x 2 ewbarnard users 4096 Mar 30 19:23 cgi-bin -rw-r--r-- 1 ewbarnard users 8 Mar 30 19:21 .htaccess -rw-r--r-- 1 ewbarnard users 747 Mar 30 19:21 index.html -rw-r--r-- 1 ewbarnard users 32 Mar 30 19:23 main.html drwxrwxrwx 2 ewbarnard users 4096 Mar 30 19:24 members
the files are owned by user ewbarnard, but the files are also part of group users. In unix there are three levels of access: user, group, and others. In the above example, the file owner may update index.html (ewbarnard has read and write permission), and everyone else (both members of the same group, and others) has read permission.
In the next section, you'll begin to realize why this is important!
File Permissions
Caveat. This does not come anywhere close to being a complete explanation of unix file permissions. We are only covering the basics you need to deal with your server!
Unix file permissions are Read, Write, and Execute, abbreviated r, w, and x. The lack of a certain type of permission is shown with a dash. So, rwx means read, write and execute permission, and rw- means read and write permission but not execute.
"Directory" means the same thing as "folder." However, with unix they're always called directories. How do you know if it's a directory or an ordinary file? On the file listing, there will be a d in the left margin just before the file permissions list. In the above example, the first three items are directories, and the last item is a directory. A dash means it's a plain file. Anything other than d or - means your ftp program might get confused.
Unix directory permissions look the same as file permissions, but they are not the same! Directories have r, w, and x permission just like files... but r, w, and x don't mean the same thing! Unfortunately, this means we need to look at r, w, and x one item at a time.
What does file read permission mean? Just what you think it should. It means that if you know where the file is, you have permission to read it. What does directory read permission mean? Again, pretty much what you would expect. It means you're allowed to scan the directory, to find out what files it contains, and anything else known about each file - when it was created, how big it is, what its permissions are, and so on. So far, so good.
What does file write permission mean? It means you can edit the file; it means you can append to the file; it means you can truncate the file. It does not mean that you can delete the file! Can you see why? To delete the file, is to remove its directory entry. The delete operation requires directory permission, not file permission. It's the same with renaming a file... renaming or moving a file requires write permission for the directories involved. Unix doesn't care if you can even read the file, so long as you have the right directory permission.
Why do you care? When you begin working with CGI scripts, the above becomes terribly, horribly, sadistically significant. But we'll explain that in a bit.
I pretty much just explained what directory write permission means. If you do not have directory write permission, you can not create a file in that directory. Even if you can edit the file, you still can't delete it!
What does file execute permission mean? It means that - in theory - the file can be treated as a self-contained unix program. It might be a "real" program like ls or cp, or it might be a text file such as a php or perl program. Without the necessary x permission, unix will refuse to recognize it. In the case of a CGI script, you'll see a 500 error.
Directory execute permission, however, means something entirely different. You don't "execute" a directory. That is, you don't attempt to run it as a computer program. What else can you do with a directory? We already have scanning and updating the directory covered - that's directory read and write permission. What's left?
If you want to scan a directory to see what's there, that's directory read permission. But what if you already know what you need? You want main.html; you don't need to go looking for it. You just want it. If that directory has execute permission, you can have it. If it doesn't, you can't. Read permission allows you to look around and be nosy; execute permission allows you to have the file you need.
Why in the world do you care about the difference? Because of how your server admin set up your server. I'll explain, but there's one more thing we need to cover first.
Three Digits
Each file or directory, then, has three permissions (rwx) each, for the file owner (user), group, and others. That's why a unix directory listing shows three sets of permissions: drwxrwxrwx for a directory, and -rwxrwxrwx for a plain file.
Unix is all about abbreviations. Vowels are never used when something unpronounceable will do. The first letter will often be used in lieu of the entire word. cp stands for copy, od stands for octal dump, yes stands for you're going to be sorry you asked.
So. If you can get a letter to stand for a word, why not have a digit stand for a series of letters each standing for a word or phrase? To a unix person, that's called efficiency, and therefore the very height of coolness.
One of the coolest things in unix, is file permissions. (I'm just giving the tip of the iceberg, you will recall... if you happened to have observed how directory set group id changes from s to S with an nfs mount, you'd begin to appreciate ultimate coolness. However, we'll leave total coolness for another day.)
Read permission is assigned a value of 4. Write permission is 2, and execute permission is 1. Thus rwx is 7, rw- is 6, r-x is 5, r-- is 4, and --x is 1. When you slap user, group, others' permissions together, you have truths such as rwxrwxrwx is 777, rw-r--r-- is 644, rw-rw-rw- is 666, rwxr-xr-x is 755, and rwx--x--x is 711.
But, what does this mean??? Finally, I can explain!
Apache Server
On your server, you are one of the users. This makes sense, right? You have an ftp login, and the files in your domain are (mostly) owned by yourself - that is, they're owned by that ftp login. On a server, the "group" concept really doesn't matter. Either everybody else is in your group, or nobody is. But it's unix, and that means the "group" part of the permission stuff is still there.
Meanwhile, the Apache server itself, is a different user. The server itself owns its own files - and does not own your files. In the choice of "user, group, others", the server falls in the "others" category. (With one important exception; I'll get to that exception in just a bit!)
When a surfer blows on in to look at your awesome web page with 300k of fast-loading graphics, what actually happens? That surfer doesn't actually log on to your server. Instead, that surfer's browser sends a request to the Apache software which is logged in to your server. More precisely, the Apache software resides on your server, and listens for requests.
When Apache sees that surfer's request, it figures out which of your files to grab, and feed to that surfer. Can Apache grab everything you've got? No, it can't. First, it's restricted by the server configuration files, and by your .htaccess files. Second, it's restricted by your unix file permissions. We'll ignore that first restriction for now, and remain focused on the second.
If the file permission is 644 (-rw-r--r--), that means that "others" can read the file but not overwrite it. Apache is one of those others. What about the directory? Every file is in some directory, and therefore directory permission always counts. If the directory permission is 711 (drwx--x--x), the server can grab the file. If the directory permission is 700 (drwx------), the server can't touch it.
CGI Scripts
Until now, everything works exactly as you'd expect. You create and upload your files; surfers surf them; all is well. Until, that is, you install a CGI script.
Again, you need to think of your server as a shared departmental computer. Do you want your colleagues to be able to trash your files any time they have a mind to? Of course not. What about the ones who are not your colleagues? Do you want to allow anyone who has an account on that computer, to edit your personal files? Probably not.
On today's servers, this might sound silly. Especially on a dedicated server, where the only person there, is you. On a shared server (i.e., a virtual host), security is tight. Most people are limited to ftp access, and they wouldn't know how to find your files if they wanted to.
What's important is the mind set. If you think like the departments did a generation ago, you'll see what I'm getting at. Do you want the world at large to be able to overwrite your files?
Yes, actually, you do!
You see, your CGI scripts fall in the category of "the world at large." It's not you running the script. Sure, you go to the URL, click the button to update, but it's still not you!
Do you see why this is so? Your browser (which is showing you the button to click) is not logged into your server. You are a surfer. Your browser sends a request over to the Apache server software residing on your server. It's Apache who figures out that you wish to update one of your own files.
From a unix standpoint, however, you are not Apache and Apache is not you. You are two different users on the same computer. You're not even part of the same departmental work group.
Can you see what this means? You're probably several steps ahead of me at this point... but don't worry, it will get tricky in just a moment!
Suppose your CGI script needs to update your index.html page. Your file has 644 permission:
-rw-r--r-- 1 ewbarnard users 747 Mar 30 19:21 index.html
That means you (the owner) can update the file, but the world at large (including the Apache server!) can not. You have a permissions problem. You need to change the file permissions to 666 (-rw-rw-rw-). Any file that your script needs to update, must have 666 permission.
Caveat. There is an important exception to the above rule, which I will explain a bit later.
What if your CGI script needs to create, delete, or move files? Create, delete, and move are directory operations. That means the directory must have 777 permission. If your directory has 755 permission (drwxr-xr-x), the script will fail to create the file. And, chances are, it will not tell you that it failed, or tell you why it failed.
Did you catch that? If the script does not have directory write permission, it won't even know that something went wrong! It will blunder merrily along, and you'll have no idea something is broken.
Would you believe that you now know more about how unix files work, than most script writers? Based on my own observations, it's true!
CGI-Created Files
You can create a file that the CGI script can't touch. If you want the script to be able to read but not write the file, that's 644 permission. You own the file, and have read/write permission - that's the 6. The script falls in the "others" category, and thus has read-only permission - that's the second 4 of the 644.
In the same way, the script can create a file that you can't touch. If the script owns the file, and the file has 644 permission, you can read but not write the file. It's a two-way street. You and the script are two different users on the same computer. You can each read the other guy's files, and can update your own.
Finally - finally - we can wallow in the delicious subtleties of unix, which will bite you good and hard. Even some of the server admins don't seem to understand this!
Once again, consider your colleague in some other department. You and she are using the same computer. You need her to drop a copy of one of her files into your directory. Therefore, you give her an open directory like this:
drwxrwxrwx 2 ewbarnard users 4096 Mar 31 12:40 tmp
777 directory permission means that she can create a file in your directory, or copy a file into your directory, which amounts to the same thing. After she's dropped the file into your tmp directory for you, your directory might look like this:
drwxrwxrwx 2 ewbarnard users 4096 Mar 31 12:42 . drwxr-xr-x 5 ewbarnard users 4096 Mar 31 12:40 .. -rw-r--r-- 1 gopher squid 8 Mar 31 12:42 info.html
(In unix, . refers to the current directory, and .. refers to one directory up - that is, the current directory's parent directory.)
Do you see the problem we have here? gopher was kind enough to provide me the file info.html. I can read the file, but I can't update it! The file has 644 permission, which would be just fine if I owned the file.
Do you see that I can rename, copy, or delete the file? Rename, copy, and delete require directory write permission, and I have plenty of that. Well, copy also requires that I read the file - but I can do that. I just can't edit it!
So... what can I do? I can't change the file's permissions, because I don't own the file. That's right! The file is in my directory. But I can't touch it (except to delete it), because I don't own it. Only the file owner (or the server admin) can change its permissions.
A generation ago, this was easy. Give gopher a call, and ask her to open up the file. No problem; it literally happened all the time. But what if the file owner is your CGI script? Who are you going to call?
Hmm... well, let's think about this. Do you actually care? Are you supposed to be editing that script's files? Probably not - in which case, just leave the file alone, and everything will be fine.
Interlude
Well. This was sadistically boring, as promised. Let me remind you of the important points, before we proceed to the main event. You'd be amazed at how many server admins don't really understand the stuff you're wading through here!
For a CGI script to find, create, delete, or move files, the script needs suitable directory permission. This usually means that if the script is doing stuff in the directory, the directory permission needs to be set to 777. If stuff is mysteriously failing to happen, make sure the directory is where the script thinks it is, and that it's set to 777.
For a CGI script to be treated as a script, the script itself needs execute permission, i.e., 755.
If a file needs to be updated (e.g., a data file), the file must be owned by the script, or have 666 permission. If a file needs to be created, it's the directory which must have 777 permission. If the script creates the file, the script owns the file, so the actual file permission won't be an issue (until we hit the main event!).
If you need to make all of your files and directories script-accessible, log into your server via telnet or ssh, and execute the following command:
find . -exec chmod ugo+rw {} \\;
That command will take care of the current directory and all subdirectories.
The Main Event
Finally! The Event has arrived! This is where things get messy, and this is where I get the calls for help.
What happens when you change servers, or restore all of your files from a backup copy? Perhaps your webspace provider put you on a different server, or perhaps you moved the whole shootin' match to a different webspace provider. Perhaps the new server admin took care of the move for you. He took care of the search and replace, to get the new pathnames correct.
In every one of these cases, we have the same problem, and it's a messy one. Do you see the problem yet? I'll be quite pleasantly surprised if you do. Many server admins don't see the problem either!
This problem is not the admins' fault, as you'll see. The admins are merely caught in the middle. The problem is not really the programmer's fault either, by the way. The problem is just a natural outcome of how unix works.
Do you recall that file info.html owned by gopher? The file with 644 permission? Let's assume that Apache is actually user gopher. In other words, your CGI script owns the file. That file can be edited by your script, so we have no problem.
Now for the Main Event. Let's just suppose that somehow that file magically became owned by you rather than by the script. It still has 644 permission - but suddenly you can edit the file, and your CGI script can not. If the script needs to edit the file, can you see how this can be a problem? You've swapped horses in midstream. You've yanked the rug out from under it. You changed the rules in the middle of the game. Your script might not recognize what just happened - and therefore you'll have no clue that something is wrong.
Give the file 666 permission rather than 644 permission, and all is well. Do you see why this is so? With 666 permission, the script can edit the file regardless of who actually owns the file.
Do you see the problem? If you somehow magically take ownership of the script's files, the script is screwed. It's your script, so that means that you just screwed yourself!
So. How do you screw yourself in such a fashion? You change servers. It's that simple.
How you changed servers, or why you changed servers, doesn't matter in the least. Perhaps your server admin upgraded you to a brand-new dedicated server. Perhaps you changed ISPs. That does not matter - it's the move itself that killed your script. You will have the same problem, by the way, if you had to restore your files from a backup. Disasters happen; that's why we make backups; recovering from the disaster can kill your script.
How do you make a backup? You make a copy of all your files and put them somewhere. Right? If you can't read the file, you can't make a backup copy of that file. This makes sense, right?
What if the CGI script owns the file? As long as you can still read the file, you can make the backup. That's what's important. So far, so good.
What if we move servers, though? How exactly do you move servers? What you do is, make a backup copy of all your stuff. Then, you unload all that stuff onto your new server. You are restoring your stuff from a backup. Sure, it's a new server, but from a unix standpoint it's a plain ole file restore.
When you do the file restore, you're screwed. It's that simple! And I can guarantee you there are plenty of server admins out there who will happily move your stuff, not realizing that they're killing your script in the process.
What just screwed you? File ownership! When you restore the files to your account, the files become owned by you. Because they're suddenly yours, you're screwed.
Remember info.html? It has 644 permission - which means that only the owner can edit it. When it was owned by the script, all was well. But now that you own it thanks to the server move, the script is broken.
Okay, so you're screwed. How do you fix the problem? You can go in with ftp, and individually change the permissions on each directory, and each file, that's affected. Or, you can use the find instruction that I showed you above.
Caveat
Many servers run their CGI scripts precisely as I describe above. That is, you and your script are two entirely different persons, so far as file ownership is concerned.
But... what if your script pretended to be you? Wouldn't that simplify things? Of course it would. Never again do you have to worry about those subtleties of file ownership. What's yours is yours, and what's the script's is yours too.
But... how does the script become you? That's a very tricky thing to do in unix. What's important here, is for you to know when this is happening. Why is it important? Because it makes such a tremendous difference to how you set up your file permissions.
Okay, so you need to know if your scripts become you, or not. How do you find out? That's easy - you ask your server admin. What's difficult, is knowing what question to ask. There is no single correct question to ask! Like I said, this is a tricky issue in unix, and therefore truly cool.
Try asking the question this way: If I upload a file a file for my CGI script to edit, does the file need to have 644 permission, or 666 permission? Unfortunately, your admin might miss the point of the question. So, at the same time, ask this second question: If my CGI script creates a file, is that file owned by my ftp user id, or is it owned by the Apache server's user id? If your admin has no idea what you're asking, consider that a clue.
This bit of trickiness has a number of different names in unix; the most common are SuExec and CGIWrapper. All of these mean that your CGI script pretends to be you.
Remember that if you change servers - such as upgrading to a dedicated server - the answer to your question will probably change, and that means your file and directory permissions must change too!
Surely I have convinced you that using this trickiness would save a whole lot of hassle? Why do many unix server setups not use this trickiness? It's not a question of convenience, but one of security.
Server Security
It's time to back up, and think like your server admin. Do you want your server hacked? Of course not! Neither does your admin. Do you want some other idiot on your server bringing the thing to its knees, so that your domains become unavailable? Of course not! Neither does your admin.
Therefore, on a shared server, your admin will do various things to protect you from your colleagues, and to protect your colleagues from you. This is unix, and that means you're all sharing a departmental computer.
There are two basic approaches to CGI script security, and you already know what those are. The first approach says the script is not you, and the second approach says the script is you. If you understand that, you understand CGI security!
Not You
The first approach means you need to have your files and directories wide open. For your CGI script to create a file, that file's directory must have 777 permission. You can't get more open than that - there's nothing wider. To edit a file, the file must have 666 (or 777) permission. Again, that's as unsecure as it's possible to be in unix.
Why is this a good thing? Because you only need to make certain files and directories vulnerable. You can keep the rest of your files untouchable. Your CGI script therefore has very clear-cut boundaries. It cannot create new files or edit other files even if it wanted to. It can only touch the places where you have explicitly given it permission.
Using this approach, some of your stuff is wide open, but the rest of your stuff is untouchable. This is a standard (and secure) approach. Of course, you could do something really dumb like this:
find . -exec chmod ugo+rw {} \\;
What have you done? You've just made all of your files completely vulnerable. Your script can edit them, but so can anyone else who happens to be on your server. On a shared server, this is a problem. On a dedicated server, this is not a problem - until someone figures out how to hack your script.
Is You
The other approach is for the script to pretend to be you. This is the more common approach for a shared server. Do you see the primary reasons?
I'm sure you see the first reason... if you're on a shared server, you're likely to be less experienced. You'd probably rather not be bothered with the complexities of unix ownerships and permissions. Your CGI script pretends to be you, and everything works the way you would expect.
The other reason is more subtle. You're forced to be a good neighbor. Have you considered messing with your neighbor's files, either by accident or on purpose? Can you pop into the server via ftp, and mess with the other guy's files? Of course not - you don't own them. And now, neither can your CGI script, because the script is no more (and no less) privileged than you are.
Consider the other situation - that is, when the script does not pretend to be you. It does not pretend to be your neighbor either. If you both run CGI scripts, you both need to leave certain files and directories wide open. And, therefore, you are vulnerable to each other.
Therefore, rather than leave you vulnerable to your neighbors, your server admin will probably configure the Apache server in such a way that your scripts pretend to be you. Your neighbor's scripts pretend to be him, and therefore can't mess with your stuff.
On a dedicated server, you don't need to worry about malicious neighbors. You're the only user of the server. Therefore, the first approach makes more sense. The script does not pretend to be you, and therefore you can limit its access to only certain files and directories.
On a shared server, your greater danger is from your neighbors - remembering that your neighbor could be a victim of a hacking expedition. It makes more sense for your scripts to run as if they were you, sidestepping those permissions problems, and keeping you out of your neighbor's stuff.
Warning
Unix and arrogance go together. Larry Wall, the creater of Perl, calls it hubris, and declares it to be a mandatory trait. A generation ago, the standard answer to any unix question was, "read the man page." It may take you several days to figure out which man page to go read; that fact was both assumed and expected. No self respecting unix guru will waste his time with someone incapable of figuring out which man page to read; and all unix gurus are self respecting.
Don't be put off by the arrogance and condescension of this article. I was portraying the atmosphere endemic to all unix discussions. You do not, of course need to put up with such crap. Instead, just read the man page.

Setting Up an iWeb Article

Thu, 2 Jul 2009 08:21:36 -0500

This is the “template” article to use as the starting point for each new post.