The Quick and Dirty on iNodes at Hostmonster
This sticky topic is an attempt to consolidate information provided by the
overwhelming response of our community to the recent enforcement of file
limits at Hostmonster.
It is a closed topic to help keep everything streamlined. If you wish to contribute to the conversation, please post your comments here. If you have a script that can help with file count monitoring or anything else useful, please pm me, or post your suggestions in the linked thread above. If deemed appropriate, it will be moved to this thread.
The purpose of this thread, as stated, is to help consolidate information you might need that will help you learn more about the iNode limits and how to monitor them for your Hostmonster account. It is for this purpose that the thread is closed.
This sticky will cover a few things:
- The stated iNode limit that HM is imposing on their customers
- A brief history of HM and our thoughts on why the need for this policy(?)
- How to find, download and read the userquota file
- Other useful scripts that you can use to help monitor and manage your iNode count
Contributors will be listed.
1. OFFICIAL iNODE LIMITS FOR ALL ACCOUNTS AT HOSTMONSTERThe official response from Hostmonster on iNode limits is 50,000 iNodes per account. This limit includes directories, emails, and everything else inside your account's home directory (/home/account/*).2. BRIEF HISTORY OF HM AND THE REASON(?) FOR THIS POLICY
An iNode is basically a pointer to the location to everything in the Linux system. This includes directories, emails, backup files, basically everything that is inside your home directory.
To be filled in later ...3. USERQUOTA
There is little you can do that is offered by Hostmonster at the moment to help manage your file counts. One thing that Hostmonster provides is a file called "userquota". This file is located inside your home directory in the tmp folder, along with other stuff like your cpu_exceeded log and your slow mysql queries folder.4. OTHER SCRIPTS THAT ARE USEFUL TO MONITORING AND MANAGING YOUR iNODE COUNT
You can either download this file via FTP or File Manager, or you can read it directly from inside File Manager by navigating to the folder, /home/account/tmp and opening the userquota file inside.
This file lists all the directories inside your account and gives the current count of files inside each directory. The file structure is like this:
The number is the file count for that directory. It lists folders by the number of files. Highest numbers first. What is interesting, is this count also includes all the files necessary to run and manager your account.Code:469 : /home/account/path/to/folder
While this file is a good place to start looking for where you need to start trimming your account, this file is not updated very often. In fact, I got rid of a huge subdomain for a family member and reduced my count by almost 20,000 files, but several hours later, my userquota still shows these files as existing. I think I remember reading this file updates every 4 hours, but my experience says it's longer than that.
What would be interesting to find out is how many files are associated with a brand new account, with no data uploaded. Would anyone be willing to check this? It would be good to see what the starting point is, since part of our 50,000 count includes the initial setup files for a basic cPanel installation.
There have been several suggestions and offerings as to how to find out the number of files in your account. They range from command line (SSH) commands to php scripts.
We would like this section to keep growing, at least until Hostmonster offers a more robust solution to helping us manage our files. This section might also contain information on how to change your thinking in managing websites, perhaps storing static data in a zip file, unzipping this file only when needed.
For instance, I went from 19000 + directory to one that contains less than 20 files/folders. I have used a combination of cron, php scripts and zip files to help reduce my file count, which is entirely different that what I was used to doing.
Another option might be to try combining all your javascript files into one file, all your css files into one file, using image sprites instead of multiple smaller image files (more on sprites later) and other things to help reduce file count.
Here are some other options:================================================== ======
Command Line OptionsUsing SSH, here is a way to get a quick count of your files, log into your home directory via SSH and run the following command:
Code:find . | wc -lThe above command gives you a count of all directories and files inside your home directory. It automatically recursions into all folders. If you want *just* files:
Code:find . -type f | wc -lIf you want to specify a directory:
Code:find ./public_html/some/directory -type f | wc -lOther scripts exist, and will be added here over time. This should be a good place to start, though and should give you an idea of the work ahead of you as you try to come within the limits of Hostmonster's latest policy ...UPDATE ON FIND COMMANDS USED ABOVE
Added June 25, 2009 - shadmegoBecause the find command impacts server performance too much to be useful, I recommend not using the above commands at all when trying to find the number of files in your directory.
With the recent shift in Hostmonster policy in allowing accounts with up to 200K files (major kudos to Hostmonster), there may well be little need to check this any longer.
Added Aug. 5, 2009================================================== ======I have not been able to verify the supposed 200,000 inode count. There used to be a thread discussing this where an employee of HM said they raised the limit, but that thread has mysteriously vanished. If anyone wants to try and verify this new count, I can update this thread accordingly.
================================================== =======If you still want to try and keep tabs on your file counts though, I would offer this new command line option, that combines "ll" with a few "grep" options to get an accurate file count.
Notes:
I don't know now much less this impacts the server, though I suspect it's acceptable because "ll" doesn't really do anything but create a "l"ong "l"isting of all files/folders. grep is used to strip out everything you don't want.
Here is the command:
Code:ll -R | grep -v "\(total \)" | grep -v "\.\?[/]$" | grep -v "^$" | wc -lA bit of background:
We discussed the ll command. The -R switch is for a recursive listing. The grep commands strip out anything that isn't a folder or file and the word count (wc) with the -l switch counts individual lines.
If anything can think of a way to condense this command, please let us know. I will post updates to this as they come in.
PHP OptionsHere is a PHP script to display the contents of your userquota file in a web browser. Simply change username to your account name and navigate to the file in your web browser. - sjlplatPHP Code:<?php
$path = "/home/username/tmp/userquota";
$file = file($path);
$line = array_reverse($file);
$lines = count($line);
echo "<ul>";
for ($i = 0; $i < $lines; $i++) {
if ($i == 0) {
echo "<li style=\"font-weight:bold;\">$line[$i]</li>";
}
else {
echo "<li>$line[$i]</li>";
}
}
echo "</ul>";
?>Added June 17, 2010 by stenyak================================================== =======
I recently had my account locked due to excessive number of inodes.More to follow.
The usequota file is updated once a day (or on demand, if you're having problems and are in contact with support people).
The "find" command is hd and cpu intensive, but it has some sort of internal cache that speeds up the process the next times. So "find" should not be feared at all.
If you're still unsure to use it, first do a find > myList, and then replace find with "cat myList".
This is a command that returns the number of inodes in current dir, recursively:
This is a command that returns the top-10 inode eaters in current directory:Code:find |cut -d "/" -f 1|uniq -c
This is a command that returns the top-10 inode eaters in next level of subdirectories:Code:find |cut -d "/" -f 2|uniq -c |sort -n |tail
This is a command that returns the top-10 inode eaters in second-next level of subdirectories:Code:find |cut -d "/" -f 3|uniq -c |sort -n |tail
If someone is kind enough to add this commands to the aforementioned thread...Code:find |cut -d "/" -f 4|uniq -c |sort -n |tail
~regards
