![[Photo of the Author]](../../common/images/Atif-Ghaffar.jpg)
original in en Atif Ghaffar
Atif is a chameleon. He changes his roles, from System Administrator, to programmer, to teacher, to project manager, to whatever is required to get the job done.
Currently he is working as the Internet Development manager at 4unet and is looking for talented people to hire.
More about him can be found at his homepage
In this Article we will setup a scalable ISP service based on LDAP and Linux. It will touch many of the issues and questions asked. We will then use a tool called ISPMan to manage it.
To avoid the unnecessary length with the textual examples, I have embedded them in textarea widgets. If you really want to print this page, then save it locally and change textarea to pre tags.
Perhaps this perl-liner will do the trick.
perl -pi.bak -e 's!textarea.*?>!pre>!g' filename
Building an ISP and then managing it is tricky, especially if you want to make it mostly automated, highly available and scalable.
Running an ISP requires a team of system administrators, dedicated to the running of machines, creating accounts, managing web-sites,
trouble shooting, running a help desk etc.
Most of the time, the help desk has little or no control on the setup. Most of the help desks I have come in touch with desperately need help.
LDAP is an excellent directory. It can manage not only the username/password of users but much more about users and resources. As we go along we will see how LDAP helps to manage a lot of things centrally.
ISPMan is the software, I wrote, so people from the IT department dont bug me when they need to create a new domain, set up a new webserver or a change a DNS entry. ISPMan is open source software and is available at http://www.ispman.org. I wont try to explain more about ISPMan itself but the concepts behind it. You can then feel free to try it out and enhance it.
This ISP will provide DNS, mail, webmail, web hosting etc.
The idea is that a customer comes and wants to host domain "exampledomain.com".
This domain is created centrally by just a few clicks and a lot of magic is managed behind the scenes including setting up DNS, setting up the virtual server for mail and a virtual webserver.
The customer gets one ftp username to access his/her web/ftp server. Any number of users can be created within the domain for email access. Users within the domain may or may not get web space.
There is also the issue of providing internet access. This could be a rather straight forward process or it can get really complicated, so we wont be touching it in this article :).
Every one wants a website and their own domain's email addresses. Email server have been strangely tied to the system accounts for emails. I dont like it.
The problem arises on multiple levels when you want to manage [email protected] and [email protected] etc.
There is a lot of unneccesary mappings etc to be done. Hopefully future softwares will be written with domain names in mind.
For example, Cyrus, an excellent IMAP server. Cyrus manages mailboxes (not users). It only allows one mailbox to have a name "aghaffar". If I have another customers from domain "linuxrus.com" who wants a username "aghaffar", then I am out of luck. I would have to create a mailbox by another name and map the user to that mailbox. Another trick would be to create a user "aghaffar.linuxrus.com" but Cyrus used "." as the mailbox delimeter, so cant use that one, neither can I use "[email protected]".
Also note that not all of us live in US. A lot of examples in some mailing lists suggest user1@domain1 and user1@domain2. They expect all domain names to end in .com. So we have to keep track of "username" "domain" "TLD"(Top Level Domain).
Our design will take all this into account.
So instead of creating a user called "aghaffar", we will create users in this form. "username_domain_tld".
I dont really remember why I used "_" as the delimeter, but "." was not available because of Cyrus, and there were other problems with other delimiters, for example "&" is a bad one both for shells and URLs.
This is a list of software that I was able to use nicely together. You may want to use another software if you wish and if it works for you.
Our directory is based on domains. There are domains, domain users, domain services etc.
For example, a user can only exist in a domain (except some system users such as LDAP admin, Cyrus admin etc)
domain branch has information about users of that domain, the DNS data of the domain, the http data of the domain etc etc
Example
Here we defined a branch for domain "developer.ch", this branch has sub branches for users, dnsdata, and httpdata.
In this example, we defined a uid, gid, homeDirectory etc for the domain, cause we want only the user "domain.tld" to access via ftp.
For example, if the customer who owns developer.ch wants to upload files to her directory, then she will login as user "developer.ch" and the appropriate password to login to the ftpserver which will hopefully chroot to homeDirectory.. etc .. etc.. more about that later.
For other users, we dont define uid, gid, etc cause we dont want them to login via ftp.
You can find a complete LDIF example file here (this file may be outdated cause its from a production machine, and I have added features etc in the new design) or use this link if you want to show it to your boss (again a bit outdated)
Another great thing is that we do not have to create any system accounts in /etc/passwd, /etc/shadow etc or manage NIS etc.
All accounts are manged in LDAP and authentification is done directly from LDAP.
For this trick we take a lot of help from PAM (Pluggable Authentification Module) pam_ldap.
PAM allows you to define which module should take care of the authentification, authorization etc.
For example, my /etc/pam.d/imap, /etc/pam.d/pop and /etc/pam.d/proftpd files say
#%PAM-1.0 auth sufficient /lib/security/pam_ldap.so account sufficient /lib/security/pam_ldap.soNow all imap/pop3/ftp authentifications are handed to the ldap server. So user aghaffar_developer_ch gets authentificated and gets his mails even though he does not exists on the system's passwd or nis records.
DNS does not at the moment have an LDAP back end, and perhaps its not a good idea to have an LDAP backend for DNS except for Dynamica DNS which is used in conjunction with DHCP. Any way, we use LDAP to store information about DNS and then extract it to create DNS zone files. This allows us to manage everything centrally and from a secure machine, while not making any changes to DNS.
The DNS entries in the LDAP look like this
dn: ou=dnsdata, domain=4unet.net, o=ispman
domain: 4unet.net
ou: dnsdata
objectclass: top
objectclass: domainrelatedobject
objectclass: posixAccount
uid: 4unet.net
uidNumber: 2000
gidNumber: 1000
homeDirectory: /home/4unet.net
userPassword: {crypt}XXffGGHH
loginShell: /bin/true
|
The dnsdata branch definition. It also defined a posixAccount(a user) with the same name as the domain name. This user is sort of webmaster, who can log in via ftp and upload files to the designated areas etc. |
dn: cn=soarecords, ou=dnsdata, domain=4unet.net, o=ispman cn: soarecords primary: ns1.4unet.net ou: dnsdata retry: 1800 rootmail: dnsmaster.4unet.net domain: 4unet.net minimum: 432000 objectclass: top objectclass: domainRelatedObject expire: 1209600 refresh: 21600 |
This defines the SOA records for the DNS for 4unet.net. A script is responsible to extract the values and format them into a DNS SOA record to be included in the zone file |
dn: cn=nsrecords, ou=dnsdata, domain=4unet.net, o=ispman domain: 4unet.net cn: nsrecords ou: dnsdata objectclass: top objectclass: domainRelatedObject record: @,ns1.4unet.net record: @,ns2.4unet.net |
And these are the NS records.
A script will grab all the record attributes, and split them by character "," and get the target and nameserver to be added to the zonefile |
dn: cn=mxrecords, ou=dnsdata, domain=4unet.net, o=ispman domain: 4unet.net cn: mxrecords ou: dnsdata objectclass: top objectclass: domainRelatedObject record: @,10, mx1.4unet.net record: @,100, mx2.4unet.net |
Same as above, but MX records this time.
These records also contains the priority field which is accordingly adjusted in the MX records of the zonefile |
dn: cn=arecords, ou=dnsdata, domain=4unet.net, o=ispman objectclass: top objectclass: domainRelatedObject domain: 4unet.net cn: arecords ou: dnsdata record: ns1, 193.247.80.43 record: ns2, 193.247.80.44 record: @,193.247.80.43 record: @,193.247.80.44 |
These are the A records, simply host, ip address mappings |
dn: cn=cnames, ou=dnsdata, domain=4unet.net, o=ispman objectclass: top objectclass: domainRelatedObject domain: 4unet.net cn: cnames ou: dnsdata record: ftp, www record: mail, www record: *, www |
And the CNAMES or Aliases for the hosts |
You need to have proftpd with the LDAP module installed. If you are going to run ftp with virtual servers etc, and you think it will be quiet often busy then you should run it standalone instead of from inetd.
So comment the ftp lines from your inetd.conf and reload the inetd daemon.
Create a group with the gid 1000 called ftponly. We will give this gid to all domains.
edit your /etc/pam.d/proftpd as shown in LDAP authentification section above.
your /etc/proftpd.conf should look something like this.
To run proftpd, you can type /usr/sbin/proftpd and to kill it, killall /usr/sbin/proftpd
Compile and install Cyrus SASL and imapd, and the UW-IMAP client's c-sdk. The IMAP, client's sdk may already be installed on your system. Try first rpm -aq | grep imap. If in doubt, compile and install a newer version anyway. create a system user called cyrus and group mail, follow the installation instruction and test that you imap server is working. Once it is working simply set /etc/pam.d/imap as shown above in the LDAP authentification section.
you will need to create a user called cyrus or whatever admin your gave to /etc/imapd.conf in the LDAP directory.
For example if your cyrus admin is called "cyrus", then you may have an entry like the one below in your ldap directory.
dn: uid=cyrus, ou=admins, o=ispman cn: Cyrus Admin sn: Cyrus objectclass: top objectclass: systemadmins uid: cyrus userpassword: XXDDCCYY ou: adminsISPMan will make these entries for your during setup.
Postfix is very friendly to LDAP. You can make virtual domain and virtual user lookups directly in LDAP so you dont have to specify which domains you should recieve mails for.
For example, if a mail arrives for domain "perl.ch", it should look in all domains and see it there is a domain called "perl.ch", if there is then it should accept the mail instead of returning "Duh MX for perl.ch loops back to myself".
my /etc/postfix/main.cf looks like this
ISPMan makes it easy to manage users. Creating a user consists of two steps
ISPMan allows you to create user's mailbox to any machine in your mailfarm. For example, you may have mail1, mail2, mail3, mail4 etc, each managing 10,000 users. using the LDAP + Postfix + Cyrus combo, you can deliver mail to any of the internal machines.
For example, mail arrives for [email protected], Postfix asks ldap server to retun the maildrop for the entry that matches [email protected], and the LDAP server returns [email protected]. The mail is then routed to the machine called mail5 in your mailfarm.
Currently I am working along with some cool developers on a IMAP/pop3 proxy that runs on the frontend mail servers on pop3 and imap port and transparently redireects requests to internal machines. So your users will only know one address for example mail.developer.ch or pop.developer.ch to connect to instead of knowing which mailserver their mail resides on.
IMP is a great product for providing web mail to users.
You can set an alias of mail.* in apache's httpd.conf to point to the central IMP installation.
For example the following is from my installation
I am working on a slightly modified version of IMP that will handle the following cases.