Latest posts for tag etiopia

First day in Addis

First day in Addis Ababa, after the introductory session for this 10 days Linux training.

Interesting new quotes I picked up from the excellent presentation of Dr. Dawit:

Much that I bound I could not free Much that I freed returned to me

(I didn't manage to transcribe the attribution)

And this one for Bubulle, about translation:

When you speak to me in my language you speak to my heart when you speak to me in English you speak to my head

(sb.)

Incomplete list of questions I've been asked, in bogosort -n order:

How do I get support?
Are the configuration files always the same accross different distributions?
What is the level of interoperatibility between the various Linux distributions? And between different Unix-like systems?
Does plug and play work well when I change hardware?
Can I access NTFS partitions?
How do I play multimedia files in restricted formats?
I heard that NFS has security problems: can it be secured, or are there other file sharing alternatives?
Can I access a desktop remotely?
Can I install Linux on a computer where there's Windows already? Do I need to partition?
Can I be sure to find drivers for my hardware?

I'm happy to find that we've been successful in building more and more good answers for these questions.

2009-06-06

eng etiopia pdo

Seventh day in Addis

Setting up a mail server

Background

Some terminology:

MTA: Mail Transport Agent
MUA: Mail User Agent
MDA: Mail Delivery Agent
SMTP: Simple Mail Transfer Protocol
MX: Mail eXchange
POP: Post Office Protocol
IMAP: Internet Message Access Protocol

With SMTP you connect to a server and send two things: envelope and message.

The envelope looks like this:

MAIL FROM: <enrico@enricozini.org>
RCPT TO: <rms@fsf.org>
RCPT TO: <linus@linux.org>

The message looks like this:

From: <enrico@enricozini.org>
To: <rms@fsf.org>
Cc: <linus@linux.org>
Message-ID: <1234567@enricozini.org>
Subject: Test mail

Hi Richard,

this is a test mail.  I'm also writing
Linus to show how to send to more people.

Cheers,

Enrico

There is no authentication.

There is no encryption.

Two usual types of access control:

Outbound e-mail is normally only accepted from an internal network
Inbound e-mail is normally accepted from anywhere

The DNS is used to find the SMTP server to use to send a message:

$ host -t MX yahoo.com
yahoo.com MX 10 smtp1.yahoo.com
yahoo.com MX 20 smtp2.yahoo.com
yahoo.com MX 20 smtp3.yahoo.com

The process of sending an E-Mail:

Enrico writes an E-Mail:

From: Enrico Zini <enrico@enricozini.org>
To: Richard Stallman <rms@fsf.org>
Subject: Hello from Addis

Hi Richard,

Addis is a wonderful city, even if
it rains a lot.

Bye,  Enrico

Enrico's MUA connects to the SMTP server (for example, port 25 of smtp.aau.edu.et):

HELO enricozini.org
200 OK Hello enricozini.org
MAIL FROM: <enrico@enricozini.org>
200 OK Mail from enrico@enricozini.org
RCPT TO: <rms@fsf.org>

Here, the SMTP server performs relay control: "do we relay mail to rms@fsf.org?":

Outbound e-mail is normally only accepted from an internal network
Inbound e-mail is normally accepted from anywhere

A target address could be refused:

413 ERR I don't relay for rms@fsf.org

In this case, the destination is not local but the recipient is accepted because I'm inside the local network:

200 OK Destination rms@fsf.org
DATA
200 OK Please send message body
From: Enrico Zini <enrico@enricozini.org>
To: Richard Stallman <rms@fsf.org>
Subject: Hello from Addis
Date: Mon, 17 Jul 2006 09:49:45 +0300
Message-ID: <124372643@enricozini.org>

Hi Richard,

Addis is a wonderful city, even if
it rains a lot.

Bye,  Enrico
.
200 OK Message accepted
QUIT
200 OK Bye.

The SMTP server needs to find out where to send the message, using the DNS:

$ host -t MX fsf.org
fsf.org MX 10 mail.fsf.org
fsf.org MX 20 mail.gnu.org

So the SMTP server tries the first one and connects to port 25 of mail.fsf.org:

HELO smtp.aau.edu.et
200 OK Hello smtp.aau.edu.et
MAIL FROM: <enrico@enricozini.org>
200 OK Mail from enrico@enricozini.org
RCPT TO: <rms@fsf.org>

The destination is accepted because it's for a local user::

200 OK Destination rms@fsf.org
DATA
200 OK Please send message body
From: Enrico Zini <enrico@enricozini.org>
To: Richard Stallman <rms@fsf.org>
Subject: Hello from Addis
Date: Mon, 17 Jul 2006 09:49:45 +0300
Message-ID: <124372643@enricozini.org>
Received: by mail.aau.edu.et
  on Mon, 17 Jul 2006 09:55:53 +0300
  from 10.4.15.158

Hi Richard,

Addis is a wonderful city, even if
it rains a lot.

Bye,  Enrico
.
200 OK Message accepted
QUIT
200 OK Bye.

Now, mail.fsf.org will invoke a MDA to write the mail in Richard Stallman's mailbox.

Example of problems with mail handling:

Accepting inbound connections:
Malicious input:
- logic errors
- buffer overflows
- DoS (Denial Of Service) attacks
- Connection floods
Performing outbound connections:
Programming errors:
- Flooding of connections
Performing routing:
Unauthorised relays
Mail loops
Writing to the local hard drive:
Filling up the hard drive
Writing to the wrong files
Writing to the local hard drive as root:
In case of error or attack, any file in the system can potentially be compromised

RFC-822 is the original standard for E-mail. RFCs are standard Internet documents. Have a look at RFC documents released the 1st of April.

postfix

Common setup: "Internet site with smarthost".

More difficult to maintain: "Internet site".

A smarthost is a machine that will relay e-mail for you.

Questions asked with "Internet site with smarthost":

Mail name: aau.edu.et (name used to publicly identify the mail server)
Smarthost name: smtp.telecom.net.et (SMTP server that will relay our e-mail)

To test a mail server::

$ telnet localhost 25
HELO me
MAIL FROM: <a@b.c>
RCPT TO: <mail@of.a.local.user>
DATA

hi
.
QUIT

By default, you find locally delivered mail in /var/mail/username.

Postfix configuration files:

/etc/postfix/master.cf: configures how all the postfix components run together (man 5 master)
/etc/postfix/main.cf: Main postfix configuration (man 5 postconf)

To rewrite addresses:

In /etc/postfix/main.cf::

canonical_maps = hash:/etc/postfix/canonical

Then in /etc/postfix/canonical you can add the rewrite rules, like::
```
enrico   enrico@enricozini.org
```
When /etc/postfix/canonical is modified you need to regenerate the index::
```
sudo postmap canonical
```
(same is when you change the alias file: sudo postalias /etc/aliases)

(see file:///usr/share/doc/postfix/html/ADDRESS_REWRITING_README.html)

Manipulating the message queue:

mailq - List the mail queue.

Example::

    mailq

postqueue - Postfix queue control

Examples::

    # Like mailq
    postqueue -p

    # Tries to send every message in the queue
    postqueue -f

    # Tries to send every message in the queue for that site
    postqueue -s site

postsuper - Postfix superintendent

Examples::

    # Deletes one message
    sudo postsuper -d 7C4D2EC0F5D

    # Deletes all messages held in the queue for later delivery
    sudo postsuper -d ALL deferred

Different mail queues in postfix:

incoming: mail who just entered the system
active: mail to be delivered
deferred: mail to be delivered later because there were problems
hold: mail that should not be delivered until released from hold

Mail logs are in::

/var/log/mail.log
/var/log/mail.err
/var/log/mail.info
/var/log/mail.warn

Mail delivery

Mailbox formats:

mbox: single file, mail separated by "From " lines
maildir: one directory per folder, one file per mail
mh: similar to maildir, but not really used

Alternate MDA: procmail: allows to filter mail automatically into different folders.

Mail forwarding with ~/.forward: allows to redirect mail to a different address: just put the address you want to send to in the file ~/.forward.

POP or IMAP server

Installation:

apt-get install dovecot

Configuration is in::

/etc/dovecot/dovecot.conf

The main thing that is needed is to enable the mail protocols you want::

protocols = imaps

Server monitoring

To make all sorts of graphs::

apt-get install munin munin-node

Example: http://munin.ping.uio.no

To compute more statistics:

anteater
isoqlog
mailgraph

Monitor system logs: logcheck:

sends you mail with abnormal log lines
It's important to customize what is normal and you do it with regular expressions

Filtering viruses and spam

clamav - Virus scanner

Virus scanning:

Postfix gives the mail to clamav that scans it and gives it back if it's clean.
Strategies for infected mail:
silently delete it
refuse the mail and send a notification to the sender
refuse the mail and send a notification to the receiver
quarantine the e-mail
refuse delivery with a SMTP error
deliver with an extra header that says that it's a virus

spamassassin - Spam filter

Spam scanning:

Postfix gives the mail to spamd that scans it and gives it back with some spam information.
Strategies for spam mail:
silently delete it
refuse the mail and send a notification to the sender
refuse the mail and send a notification to the receiver
quarantine the e-mail
refuse delivery with a SMTP error
deliver with an extra header that says that it's spam
New techniques:
greylisting: when you receive a mail from a host you've never seen before, refuse it with a temporary error, and accept it the second time (after some time delay). Spammers normally don't retry, and implementing retry would increase their cost of sending e-mail.
crossassassin: if more than some amount of your users receive a mail with the same message ID, throw it away. Sending mails with different headers would increase the cost of sending e-mail.

Man pages and sections

Man pages are divided in sections:

man man shows all the sections of the manpages
man 5 postconf shows the postconf manpage in the "configuration file" section
Normally manpages are referred as manpage(section) (e.g. postconf(5) )

Authentication and encryption with SMTP (update by Marius Gedminas)

You can have authentication and encryption with SMTP:

Cheat sheet

Setting up the client (I assume Ubuntu)

  # vi /etc/postfix/main.cf

      relayhost = [hostname.of.your.ISPs.smtp.server]
      smtp_use_tls = yes
      smtp_enforce_tls = yes
      smtp_tls_enforce_peername = no
      smtp_sasl_auth_enable = yes
      smtp_sasl_password_maps = hash:/etc/postfix/smtp_auth
      smtp_sasl_security_options = noanonymous

  # vi /etc/postfix/smtp_auth

      [hostname.of.your.ISPs.smtp.server] username:password

  # chmod 600 /etc/postfix/smtp_auth
  # postmap /etc/postfix/smtp_auth
  # postfix reload

(It would be a good idea to make the client verify the server's certificate to prevent man-in-the-middle attacks, but I haven't figured out that part yet...)

Setting up the server

  # apt-get install sasl2-bin libsasl2-modules
  # saslpasswd2 -u hostname.of.the.server -c username1
  # saslpasswd2 -u hostname.of.the.server -c username2
  ...

        these commands create /etc/sasldb2

  # echo "pwcheck_method: auxprop" > /etc/postfix/sasl/smtpd.conf
  # touch /var/spool/postfix/etc/sasldb2
  # echo mount --bind /etc/sasldb2 /var/spool/postfix/etc/sasldb2 \
          > /etc/init.d/local-sasl-for-postfix
  # chmod +x /etc/init.d/local-sasl-for-postfix
  # ln -s ../init.d/local-sasl-for-postfix /etc/rc2.d/S19local-sasl-for-postfix
  # /etc/init.d/local-sasl-for-postfix
  # adduser postfix sasl

        these commands let postfix (which runs chrooted) access /etc/salsdb2

  # cd /etc/postfix
  # openssl req -new -outform PEM -out smtpd.cert -newkey rsa:2048 -nodes \
            -keyout smtpd.key -keyform PEM -days 365 -x509
  # chmod 600 smtpd.key

        these commands create a self-signed SSL certificate

  # vi main.cf

      smtpd_sasl_auth_enable = yes
      broken_sasl_auth_clients = yes
      smtpd_sasl_local_domain = hostname.of.the.server
      smtpd_recipient_restrictions = permit_mynetworks,
                                     permit_sasl_authenticated,
                                     reject_unauth_destination
      smtpd_use_tls = yes
      smtpd_tls_cert_file = /etc/postfix/smtpd.cert
      smtpd_tls_key_file = /etc/postfix/smtpd.key

  # /etc/init.d/postfix restart

2009-06-06

eng etiopia pdo tips

Ninth day in Addis

SSH

To enable remote logins with ssh

apt-get install openssh-server

Then you can login with:

$ ssh efossnet@proxy.dream.edu.et

To verify the host key fingerprint of a machine:

$ ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub

Note: you need to verify it before logging in!

More information at http://www.securityfocus.com/infocus/1806

Example ssh usages

To log in:

    $ ssh efossnet@proxy

To run a command in the remote computer:

    $ ssh efossnet@proxy "cat /etc/hosts"

To copy a file to the remote computer:

    $ scp Desktop/july-18.tar.gz efossnet@proxy:

To copy a file from the remote computer:

    $ scp efossnet@proxy:july-18.tar.gz /tmp/

Beware of brute-force login attempts

Warning about SSH: there are people who run automated scans for ssh servers and try to login using commonly used easy passwords.

If you have an SSH server on the network, use strong passwords, or if you can it's even better to disable password authentication: in /etc/ssh/sshd_config, add:

    PasswordAuthentication no

To log in using public/private keys:

Create your key:
```
ssh-keygen -t rsa
```
Copy your public key to the machine where you want to log in:
```
ssh-copy-id -i .ssh/id_rsa.pub efossnet@proxy
```
Now you can ssh using your RSA key

If you use ssh often, read these:

proxy

Problems we had today with the proxy:

ssl does not work

Reason: squid tries to directly connect to the ssl server, but the AAU network wants us to go through their proxy.

Ideal solution: none. There is no way to tell squid to use a parent proxy for SSL connections.

Solution: update the documentation for the Dream university users telling to setup a different proxy for SSL connections.

Longer term solution: get the AAU network admins to enable outgoing SSL connections from the Dream university proxy.

Other things that can be done:

report a bug on squid reporting the need and requesting the feature
download squid source code and implement the feature ourselves, then submit the patch to the squid people

Browsing normal pages returns an error of 'Connection refused'.

In the logs, the line is:

1153294204.912    887 192.168.0.200 TCP_MISS/503 1441 GET http://www.google.com.et/search? - NONE/- text/html

That "/503" is one of the HTTP error codes.

Explanation of the error codes:

Reason: the other proxy is refusing connections from our proxy.

Solution: none so far. Will need to get in touch with the admins of the other proxy to try to find out why it refuses connection to our proxy, and how we can fix the problem.

postfix on smtp.dream.edu.et

Basic information is at http://www.postfix.org/basic.html.

Difference between mail name and smarthost:

The mail name is the name of the mail server you're setting up (TODO: need more details on what's it used for)
The smarthost is the name of the mail server that will relay mail for you.

Quick way to send test mails:

apt-get install mailx
echo ciao | mail efossnet@localhost

To configure a workstation not to do any mail delivery locally and send all mail produced locally to smtp.dream.edu.et:

install postfix choosing "Satellite system"
put smtp.dream.edu.et as a smarthost.

To setup a webmail: apt-get install squirrelmail (on a working apache setup).

To setup mailing lists: apt-get install mailman, then follow the instructions in /usr/share/doc.

Mail server issues we encountered

When a mail is sent to efossnet@localhost, the system tries to send it to efossnet@yoseph.org

Investigation:

"yoseph.org" does not appear anywhere in /etc or /var/spool/postfix
postfix configuration has been reloaded
postfix logs show that the mail has been 'forwarded'

Cause: the user efossnet had forgotten that he or she had setup a .forward file in the home directory.

Solution:

 rm ~efossnet/.forward

Apache

To add a new website:

cd /etc/apache2/sites-available
sudo cp default course
sudo vi course:
1. Remove the first line
2. Add a ServerName directive with the address of your server: ServerName course.dream.edu.et
3. Customize the rest as needed: you at least want to remove the support for browsing /usr/share/doc and you want to use a different document root.
sudo a2ensite course
sudo /etc/init.d/apache2 reload

More VIM

Undo: u (in command mode)

Redo: ^R (in command mode)

You can undo and redo multiple times.

To recover a lost password for root or for the ubuntu admin user

Boot with a live CD, mount the system on the hard disk (the live CD usually does it automatically), then edit the file /etc/shadow, removing the password:

enrico:$1$3AJfasjJFHa234dfh230:13343:0:99999:7:::

becomes:

enrico::13343:0:99999:7:::

You can edit the file because, in the live CD system, you can always become root.

After you do this, reboot the system: you can log in without password, and set yourself a new password using the command passwd.

Installing packages not on the CDs

To get a package for installing when offline:

apt-get --print-uris install dnsmasq
Manually download the packages at the URLs that it gives you

Otherwise, apt-get --download-only install dnsmasq will download the package for you in /var/cache/apt/archives.

You can install various previously downloaded debian packages with:

dpkg -i *.deb

Backups

There are various ways:

dump (for ext2/ext3 file systems) or xfsdump (for xfs file systems).

Makes a low-level dump of the file system.

It must be used for every different partition.

It makes the most exact backup possible, including inode numbers.

It can do full and incremental backups.

To see the type of the filesystems, use 'mount' with no parameters.

To restore: restore or xfsrestore.

tar

Filesystem independent.

It can work accross partitions.

It correctly backups permissions and hard links.

It can do full and incremental backups.

Example:

    tar lzcpf backup.tar.gz /home /var /etc /usr/local
    tar lzcpf root.tar.gz /

To restore:

    tar zxpf backup.tar.gz

faubackup

Filesystem independent.

Uses hard drive as backup storage.

Always incremental.

It cannot do compression.

Unchanged files in new backups are just links to old backups, and do not occupy space.

Any old backup can be deleted at any time without compromising the others.

It can be used to provided a "yesterday's files" service to users (both locally and exported as a read-only samba share...).

To restore, just copy the files from the backup area.

amanda

apt-get install amanda-client amanda-server

It is a network backup system.

It can do full and incremental backups.

You can have a backup server which handles the storage and various backup clients that send the files to backup to the server.

It takes some studying to set up.

To restore: it has its own tool.

Some data requires exporting before backing it up:

To save the list of installed packages and the answer to configuration questions:
```
dpkg --get-selections > pkglist
debconf-get-selections > pkgconfig
```

To restore:

    dpkg --set-selections < list
    debconf-set-selections < pkgconfig
    apt-get dselect-upgrade

If you do this, they you only need to backup /etc, /home, /usr/local, /var.

To save the contents of a MySQL database:

mysqldump name-of-database | gzip > name-of-database.dump.gz

To restore:

    zcat name-of-database.dump.gz | mysql

You can schedule these dumps to be made one hour before the time you make backups.

Scheduling tasks

As a user:

crontab -e

As root: add a file in one of the /etc/cron.* directories.

In cron.{hourly,daily,weekly,monthly} you put scripts.

In the other directories you put crontab files (man 5 crontab).

If the system is turned off during normal maintainance hours, you can do two things:

Change /etc/crontab to use different maintanance hours
Install anacron (it's installed by default in ubuntu)

For scheduling one-shot tasks, use at(1):

$ at 17:40
echo "Please tell Enrico that the lesson is finished" | mail efossnet@dream.edu.et
^D

When and how to automate

First, you manage to do it yourself
Then, you document it
Then, you automate it

Start at step 1 and go to 2 or 3 if/when you actually need it.

(credits to sto@debian.org: he's the one from which I heard it for the first time, said so well).

Interesting programs to schedule during maintanance

rkhunter, chkrootkit
checksecurity
debsecan
tiger

Important keys to know in a Unix terminal

These are special keys that work on Unix terminals:

^C: interrupt (sends SIGTERM)
^\: interrupt (send SIGQUIT)
^D: end of input
^S: stop scrolling
^Q: resume scrolling

Therefore, if the terminal looks like it got stuck, try hitting ^Q.

Problems we had today with postfix

Problem: mail to efossnet@dream.edu.et is accepted only if sent locally.

Reason:

    $ host -t mx dream.edu.et
    Host dream.edu.et not found: 3(NXDOMAIN)

Solution: tell dnsmasq to handle a MX record also for dream.edu.et:

 mx-host=dream.edu.et,smtp.dream.edu.et,50

The problem not solved with the previous solution.

Reason: postfix was making complaints which mentioned localhost as a domain name.

Solution: fixed by changing 'myhostname' in main.cf to something different than localhost.

Note: solved by luck. Investigate why this happened.

Problems found yesterday and today

there is no way to tell squid to use another proxy for SSL connections: it only does them directly
if you want to configure evolution to get mail from /var/mail/user, you need to explicitly enter the path. It would be trivially easier if evolution presented a good default, since it's easy to compute. It would also be useful if below the "Path" entry there were some text telling what path is being requested: the mail spool? the evolution mail storage?
In Evolution: IMAP or IMAPv4r1? What is the difference? Why should I care?
apt-get --print-uris doesn't print the URIs if the package is in the local cache, and there seems to be no way to have it do it.
in /etc/apache2/sites-available/default, is the NameVirtualHost * directive appropriate there? It gets in the way when using 'default' as a template for new sites.

Otherwise, one can add a new (disabled) site that can be used as a template for new sites instead of default.

the default comments put by crontab -e are not that easy to read.

2009-06-06

eng etiopia pdo tips

Edifici

Da una canzone in amarico:

"Il tuo amore è diventato vecchio

come gli edifici costruiti dagli italiani"

2009-06-06

etiopia ita life

Eight day in Addis

Useful things to keep in mind when setting up a service:

always take note of what you do
make yourself always able to explain to another person what you did
keep a copy of the configuration files before changing them, so that you can see what you changed
be always able to move the service to another computer
make sure that it works after reboot

Example use of vim block selection:

ESC: exits insert mode.
^V: starts block selection. Move the arrows to form a rectangle.
c: change. Type the new content for the line.
ESC: gets out of insert mode, and the change will happen in all the lines.

To change network configuration with config files, edit:

/etc/network/interfaces

To also setup DNS in /etc/network/interfaces, use dns-search and dns-nameservers (for this to work, you need to have the package resolvconf):

dns-search dream.edu.et
dns-nameservers 192.168.0.1 192.168.0.2

To make a router that connects to the internet on demand using a modem:

apt-get install diald

To see the path of network packets:

mtr 4.2.2.2

Basic NAT script:

OUT=eth2
IN=eth0

modprobe iptable_nat
iptables -t nat -A POSTROUTING -o $OUT -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward

What happens at system startup:

the BIOS loads and runs the boot loader
the boot loader loads the kernel and the inintrd ramdisk and runs the kernel
the kernel runs the script 'init' in the initrd ramdisk
the script 'init' mounts the root directory
the script 'init' runs the command /sbin/init in the new root directory
'init' starts the system with the configuration in /etc/inittab

To install a new startup script:

sudo ln -s /usr/local/sbin/firewall /etc/init.d
sudo update-rc.d firewall defaults 16 75

Normally you can just do:

sudo update-rc.d [servicename] defaults

To have a look at the start and stop order numbers, look at /etc/rc2.d for other start scripts and /etc/rc0.d for other stop scripts

To test a proxy, low level way:

$ telnet proxy 8080
Trying 192.168.0.6...
Connected to proxy.dream.edu.et.
Escape character is '^]'.
GET http://www.google.com HTTP/1.0 [press enter twice]

2009-06-06

eng etiopia pdo tips

Fourth day in Addis

Unix file permissions:

    drwxr-xr-x   2 root root    38 2006-07-14
    |
    +- Is a directory

    drwxr-xr-x   2 root root    38 2006-07-14
     ---
      |
      +- User permissions (u)

    drwxr-xr-x   2 root root    38 2006-07-14
        ---
         |
         +- Group permissions (g)

    drwxr-xr-x   2 root root    38 2006-07-14
           ---
            |
            +- Permissions for others (o)

    drwxr-xr-x   2 root root    38 2006-07-14
                   ----
                    |
                    +- Owner user

    drwxr-xr-x   2 root root    38 2006-07-14
                        ----
                         |
            Owner group -+

Other bits:

4000 Set user ID:
For executable files: run as the user who owns the file, instead of the user who runs the file
For directories: I think it's not used
2000 Set group ID:
For executable files: run as the group who owns the file, instead of the group of the user who runs the file
For directories: when a file is created inside the directory, it belongs to the group of the directory instead of the default group of the user who created the file
1000 Sticky bit:
For files: I think it's not used anymore
For directories: only the owner of a file can delete or rename the file

The executable bit for directories means "can access the files in the directory".

If a directory is readable but not executable, then I can see the list of files (with ls) but I cannot access the files.

To access a file, all the directories of its path up to / need to be executable.

Commands to manipulate permissions:

chown - change file owner and group
chgrp - change group ownership
chmod - change file access permissions
sudo adduser enrico www-data adds the user enrico to the group www-data.

Example setup for a website for students:

    # Create the group 'students'
    mkdir /var/www/students
    chgrp students /var/www/students
    chmod 2775 /var/www/students

    # If you don't want other users to read the files of the students:

    chmod 2770 /var/www/students
    adduser www-data students
     (this way the web server can read the
      pages)

    # when you add a user to a group, it does not affect running processes:

     - users need to log out and in again
     - servers need to be restarted

Apache:

To install apache2 without a graphical interface:

apt-cache search apache2 | less
sudo apt-get install apache2

By default, /var/www is where is the static website.
By default, ~/public_html is the personal webspace for every user, accessible as: http://localhost/~user
By default, /usr/lib/cgi-bin contains scripts that are executed when someone browses http://website/cgi-bin/script
By default, apache reads the server name from the DNS. If we don't have a name in the DNS and we want to use the IP, we need to set:
```
ServerName 10.4.15.158
```

in /etc/apache/apache2.conf (set it to your IP address)

To access the Apache manual: http://localhost/doc/apache2-doc/manual/
http://localhost/doc/apache2-doc/manual/mod/mod_access.html The access control module
http://localhost/doc/apache2-doc/manual/mod/mod_auth.html The user authentication module

To edit a user password file, use:

htpasswd - Manage user files for basic authentication

Example .htaccess file to password protect a directory:

AuthUserFile /etc/apache2/students
AuthType Basic
AuthName "Students"
Require valid-user

Information about .htaccess is in http://localhost/doc/apache2-doc/manual/howto/htaccess.html
If you need to tell apache to listen on different ports, add a Listen directive to /etc/apache2/ports.conf. Then you can use:
```
<VirtualHost www.training.aau.edu.et:9000>
[...]
</VirtualHost>
```
To setup an HTTPS website:
- Documentation is in http://localhost/doc/apache2-doc/manual/ssl/
- How to create a certificate: http://www.tc.umn.edu/~brams006/selfsign.html
- Create a certificate:
  
  /usr/sbin/apache2-ssl-certificate -days 365
- Create a virtual host on port 443:
  
  [...]
- Enable SSL in the VirtualHost:
  
  SSLEngine On SSLCertificateFile /etc/apache2/ssl/apache.pem
- Enable listening on the HTTPS port (/etc/apache2/ports.conf):
  
  Listen 443

Apache troubleshooting:

check that there are no errors in the configuration file:
```
apache2ctl configtest
```

This it is always a good thing to do before restarting or reloading apache.

read logs in /var/log/apache2/
if you made a change but you don't see it on the web, it can be that you have the old page in the cache of the browser: try reloading a few times.

To install PHP

apt-get install libapache2-module-php5
then by default, every file .php is executed as php code
Small but useful test php file:
```
<? phpinfo() ?>
```

To install MySQL

apt-get install mysql-client mysql-server
for administration run mysql as root:
- Create a database with:
  
  create database students

Give a user access to the database:

# Without password
grant all on students.* to enrico;

# With password
grant all on students.* to enrico identified by "SECRET";

More information can be found at http://www-css.fnal.gov/dsg/external/freeware/mysqlAdmin.html

To use MySQL from PHP:

    apt-get install php5-mysqli php5-mysql

Problems found today:

the apache2 manual in /usr/share/doc/manual can only be viewed using apache because it uses MultiView. So you need to have a working apache to read how to have a working apache.
chmod does not have examples in the manpage.

2009-06-06

eng etiopia pdo tips

Procedure to check if all the services of Dream University are up and running

Tenth day in Addis

If a machine blocks pings, use arping instead.

Test DHCP:

$ sudo ifdown eth0
$ sudo ifup eth0
$ ifconfig

Test the DNS:

# See if the DNS machine is on
# The network
$ ping -n 192.168.0.1

# See if the DNS resolves names
$ host www.dream.edu.et

Test the gateway:

# Ping the gateway
$ ping gateway
# Ping an outside host
$ ping -n 10.4.15.6

Test the proxy:

# Ping the proxy
$ ping proxy
# Open a web page and see if it displays
# See if it caches
http_proxy=http://proxy.dream.edu.et:3030/ wget -S -O/dev/null http://www.enricozini.org  2>&1 | grep X-Cache

Test the mail server:

$ ping smtp
$ nmap smtp -p 25 |grep 25/tcp
$ if nmap gateway -p 25 |grep 25/tcp | grep -q open ; then echo "It works"; fi
$ send a mail and see if you receive it

To do more advanced network and service monitoring, try nagios:

New useful tools seen today

wget - The non-interactive network downloader.

Special devices

/dev/null:
On read, there is no data.
On write, discards data.
/dev/zero:
On read, reads an infininte amount of zero bits.
On write, discards data.
/dev/random, /dev/urandom
On read, reads random bits.
On write, discards data.
Difference: /dev/random is cryptographically secure, but it can hang waiting for system events

Example uses:

wget -O/dev/null http://www.example.org

dd if=/dev/zero of=testdisk bs=1M count=50
mke2fs testdisk
sudo mount -o loop testdisk  /mnt

Tiny little commands

true - do nothing, successfully
false - do nothing, unsuccessfully
yes - output a string repeatedly until killed

Example uses:

while /bin/true; do echo ciao; done
Using /bin/false as a shell
yes | boring-tool-that-asks-lots-of-silly-questions

Some more shell syntax

2>&1 Redirects the standard error in the standard output
2> Redirects the standard error instead of the standard output

Some people run commands ignoring the standard error: command 2> /dev/null this causes unexpected error messages to go unnoticed: please do not do it.

What to check if a machine is very slow

See if the ram is full: $ free If it is, you see what are the fattest programs using top, pressing M to sort by memory usage.
See if there are lots of programs competing for CPU: $ top
Check if you have I/O bottlenecks: $ vmstat (but I don't know how to read it)
For a desktop on older hardware, you can try xubuntu instead of ubuntu

More VIM command mode

Command mode allows to perform various text editing functions.

You work by performing operations on selected blocks of text.

Some common operations:

y: copy ("yank")
p: paste
P: paste before
d: cut ("delete")
c: change
i: insert
`a: append
.: repeat last operation

Some common blocks:

w: word
}: paragraph
left and right arrow: one character left or right
up and down arrow: this line and the one on top or below
f letter: from the cursor until the given letter
v: selection
V: line selection
^V: block selection

Examples:

yw: copy word
dw: cut word
yy: copy line
dd: cut line
V (select lines) y: copy a selection of lines
V (select lines) d: cut a selection of lines
p: paste

The best way to learn more vim is always to run vimtutor.

Installing squirrelmail

To install squirrelmail:

apt-get install squirrelmail
/usr/sbin/squirrelmail-config and configure IMAP and SMTP.

In our case, since we use IMAPS, the IMAP server is imap.dream.edu.et, port 993, secure IMAP enabled and SMTP is smtp.dream.edu.et. 3. Read /usr/share/doc/squirrelmail/README.Debian.gz (with zless) for how to proceed with setup. A short summary: * link /etc/squirrelmail/apache.conf into the apache conf.d directory * customise /etc/squirrelmail/apache.conf for example setting up the virtual hosts, or running it only on SSL

To have different virtual hosts over HTTPS, you need to have a different IP for every virtual host: name based virtual hosts do not work on HTTPS.

You can configure multiple IP addresses on the same computer: use network interfaces named: eth0:1, eth0:2, eth0:3... These are called interface aliases.

You cannot setup interface aliases using the graphical network configuration and you need to add them in /etc/network/interfaces:

    iface eth0:1 inet static
          address 192.168.0.201
          netmask 255.255.255.0
          gateway 192.168.0.3
    auto eth0:1

This is the trick commonly used to put different virtual HTTPS hosts on the same computer.

Links

squid documentation:

Shell programming:

Performance analysis:

Setting up mail services:

http://wanderingbarque.com/howtos/mailserver/mailserver.html

2009-06-06

eng etiopia pdo tips

Fifth day in Addis

Samba

To get samba:

    apt-get install samba samba-doc smbclient

To get the Samba Web Administration Tool:

    apt-get install swat netkit-inetd

The configuration is in /etc/samba:

One [global] section with the general settings
One section per share

One could use swat at http://localhost:901/ but it does not work easily on Ubuntu.

To see what is shared:

    smbclient -L localhost

To access a share:

    smbclient //localhost/name-of-the-share

To add a new user:

    sudo smbpasswd -a username

To change the password of a user:

    sudo smbpasswd username

To test accessing a share as a user:

    smbclient //localhost/web -U yared

Documentation:

    man smb.conf

To force the user or group used to access a share:

    force user = enrico
    force group = www-data

To set the unix permissions for every created file:

    # For files
    create mask = 0664
    # For directories
    directory mask = 0775

Example share configuration for a webspace:

    mkdir /var/www/public
    chgrp www-data /var/www/public
    chmod 0775 /var/www/public

Then, in /etc/samba/smb.conf:

    [web]
       comment = Webspace
       path = /var/www
       writable = yes
       public = no
       force group = www-data
       create mask = 0664
       directory mask = 0775

Example share configuration for a read only directory where only a limited group of people can write:

    [documents]
       comment = Documents
       path = /home/enrico/Desktop/documents
       force user = enrico
       public = yes
       writable = no
       write list = enrico, yared

Print server (CUPS)

Installation:

    apt-get install cupsys

Configuration:

On the web (not enabled in Ubuntu):
```
http://localhost:631/
```
On the desktop:
```
System/Administration/Printing
```

Example IPP URIs:

    ipp://server[:port]/printers/queue
    http://server:631/printers/queue
    ipp://server[:port]/...

For example:

    ipp://server/printers/laserjet

"This printer uri scheme can be used to contact local or remote print services to address a particular queue on the named host in the uri. The "ipp" uri scheme is specified in the Internet Print Protocol specifications and is actually much more free form that listed above. All Solaris and CUPS based print queues will be accessed using the formats listed above. Access to print queues on other IPP based print servers requires use of the server supported ipp uri format. Generally, it will be one of the formats listed above."

LDAP Lightweight Directory Access Protocol

Installation:

    apt-get install ldap-utils slapd

The configuration is in /etc/ldap.

To access a ldap server:

    apt-get install gq

Various LDAP HOWTOs:

GRUB

The configuration file is in /boot/grub/menu.lst.

The documentation can be accessed as info grub after installing the package grub-doc.

Quick list of keys for info:

arrows: move around
enter: enters a section
l: goes back
u: goes up one node
q: quit
/: search

Grub trick to have a memory checker:

apt-get install memtest86+

Add this to /boot/grub/menu.lst:

title Memory test
    root (hd0,5)
    kernel /boot/memtest86+.bin

Firewall

With iptables:

    man iptables
    # Only allow in input the network packets
    # that are going to the web server
    iptables -P INPUT DROP
    iptables -A INPUT --protocol tcp --destination port 80 -j ACCEPT
    # To reset the input chain as the default
    iptables -F INPUT
    iptables -P INPUT ACCEPT

Some links:

http://www.linuxguruz.com/iptables/howto/iptables-HOWTO.html
NAT = Network Address Translation http://www.netfilter.org/documentation/HOWTO/NAT-HOWTO.html

Squid

Installation:

    apt-get install squid

The configuration is in /etc/squid/squid.conf.

To allow the local network to use the proxy:

    # Add this before "http_access deny all"
    acl our_networks src 10.4.15.0/24
    http_access allow our_networks

To use a parent proxy:

    cache_peer proxy.aau.edu.et     parent    8080  0  proxy-only no-query

Pay attention because /var/spool/squid will grow as the cache is used. The maximum cache size is set in the directive cache_dir.

Information about squid access control is at http://www.squid-cache.org/Doc/FAQ/FAQ-10.html

To check that the configuration has no syntactic errors: squid -k parse.

To match urls:

    acl forbiddensites url_regex [-i] regexp

For info about regular expressions:

    man regex

Example filtering by regular expression:

    acl skype url_regex -i [^A-Za-z]skype[^A-Za-z]
    http_access deny skype

Transparent proxy setup: http://www.tldp.org/HOWTO/TransparentProxy.html

Problems found today

Hiccups of the day:

swat does not run on Ubuntu because Ubuntu does not have inetd
swat does not allow root login on Ubuntu because root does not have a password
smbpasswd -a does not seem to update the timestamp of /var/lib/samba/passwd.tdb
cups web admin does not work on Ubuntu
LDAP is still not so intuitive to set up

Update: Marius Gedminas writes:

I think it would be a good idea to mention that running
     iptables -P INPUT DROP
in the shell is a Bad Idea if you're logged in remotely via SSH.

2009-06-06

eng etiopia pdo tips

Addis course Tasks & Skills questions

What does the command find /etc | less do?
What does the command ps aux do?
What does the command mii-tool do and when would you use it?
What does the command host www.google.com do?
How do you get the MAC address of your computer?
What can you use dnsmasq for?
What is in /etc/dnsmasq.conf?
What is the use of the dhcp-option configuration parameter of /etc/dnsmasq.conf?
What is the difference between chown, chgrp and chmod?
What would you use nmap for?
How do you check to see if a network service is running on your computer?
What does apache2ctl configtest do? When should you run it?

Consider this piece of configuration of apache:

AuthUserFile /etc/apache2/students
AuthType Basic
AuthName "Students"
Require valid-user

What does it do?

What command would you use to add a new username and password to /etc/apache2/students? (you can write the entire commandline if you know it, but just the name of the command is fine)

You created the configuration for a new apache site in /etc/apache2/sites-available. How do you activate the new site?
When do you need to add the line Listen 443 to /etc/apache2/ports.conf?
What do you normally find in /var/log/syslog, and when would you read it?
What does the command smbclient //localhost/web do?
What does the command sudo smbpasswd -a enrico do?
Where do you look for the explanation of the many directives found in /etc/samba/smb.conf?
What is the purpose of the package cupsys?
What is the purpose of the command iptables?
What is the difference between MDA, MTA and MUA?
In a normal mail server configuration, when should you accept a mail coming from outside your local network?
Suppose you are a mail software and you need to send a mail to addis@yahoo.com: how do you find out the internet host to which you should connect to send the mail?
What is the difference between man 5 postconf and man 8 postconf?
What is the different use of SMTP and IMAP?
What is a "smarthost" in the context of mail server configuration?
What does the command mailq do?
What does the command sudo postsuper -d ALL deferred do?
Postfix has four mail queues: "incoming", "active", "deferred" and "hold". What is the difference among them?
What does the package dovecot do?
In the file /etc/dovecot/dovecot.conf, what is the difference between having protocols = imap and protocols = imaps?
What happens if I put the line enrico@enricozini.org in the file /home/enrico/.forward?
Consider this list of possible strategies for handling mail classified as spam:
- silently delete it
- refuse the mail and send a notification to the sender
- refuse the mail and send a notification to the receiver
- quarantine the e-mail
- refuse delivery with a SMTP error
- deliver with an extra header that says that it's spam

What are their advantages and disadvantages?

2009-06-06

eng etiopia pdo

Third day in Addis

Believe it or not, a network that fails often is the best thing to have when you are teaching network troubleshooting.

Various tools useful for networking:

ifconfig - configure a network interface
dnsmasq - Simple DNS and DHCP server
host - DNS lookup utility
route - show / manipulate the IP routing table
arping - send ARP REQUEST to a neighbour host
mii-tool - view, manipulate media-independent interface status (IOW, see if the cable works)
nmap - Network exploration tool and security / port scanner

Examples:

    # Look at what machines are active in the local network:
    nmap -sP 10.5.15.0/24

    # Look at what ports are open in a machine:
    nmap 10.5.15.26

tcpdump - dump traffic on a network

It can be used to see if there is traffic, and to detect traffic that shouldn't be there.

Useful tip:

    # Convert a unix timestamp to a readable date
    date -d @1152841341

What happens when you browse a web page:

type the address www.google.com in the browser
the browser needs the IP address of the web server:
look for the DNS address in /etc/resolv.conf (/etc/resolv.conf is created automatically by the DHCP client)
try all the DNS servers in /etc/resolv.conf until one gives you the IP address of www.google.com
take the first address that comes from the DNS (in our case was 64.233.167.104)
figure out how to connect to 64.233.167.104:
consult the routing table to see if it's in the local network:
1. if it's in the local network, then look for the MAC address (using ARP
  - Address Resolution Protocol)
2. if it'd not in the local network, then send through the gateway (again using ARP to find the MAC address of the gateway)
Send out the HTTP request to the local web server or through the gateway, using the Ethernet physical protocol, and the MAC address to refer to the other machine.

Troubleshooting network problems:

See if the network driver works:
With ifconfig, see if you see the HWaddr:. If you do not see it, then the linux driver for the network card is not working. Unfortunately there's no exact way to say that it works perfectly
See if you have an IP address with ifconfig. If you find out that you need to rerun DHCP (for example, if the network cable was disconnected when the system started), then you can do it either by deactivating/reactivating the Ethernet interface using System/Administration/Networking or, on a terminal, running:
```
# ifdown eth0
# ifup eth0
```
If you don't get an IP, try to see if the DHCP server is reachable by running:
```
$ arping -D [address of DHCP server]
```
See if the local physical network works:
With sudo mii-tool, see if the cable link is ok. If it's not, then it's a problem in the cable or the plugs, or simply the device at the other end of the cable is turned off.
Try arping or ping -n on a machine in the local network (like the gateway) to see if the local network works.
See if the DNS works:
Find out the DNS address:
```
cat /etc/resolv.conf
```
If it's local, arping it
If it's not local, ping -n it
Try to resolve a famous name using that DNS:
```
$ host [name] [IP address of the DNS]
```
Try to resolve the name of the machine you're trying to connect. If you can resolve a famous name but not the name you need, then it's likely a problem with their DNS.
If you use a proxy, see if the proxy is reachable: check if the proxy name resolves to an IP, if you can ping it, if you can telnet to the proxy address and port:
```
$ telnet [proxy address] [proxy port]
```
you quit telnet with ^]quit.
If you can connect directly to the web server, try to see if it answers:
```
$ telnet [address] 80
```
If you are connected, you can confirm that it's a web server:
```
GET / HTTP/1.0 (then Enter twice)
```
If it's a web server, it should give you something like a webpage or an HTTP redirect.

When you try to setup a service and it doesn't work:

check that it's running:
```
$ ps aux | grep dnsmasq
```
check that it's listening on the right port:
```
$ sudo netstat -lp
```
check that it's listening from the outside:
```
$ nmap [hostname]
```
check for messages in /var/log/daemon.log or /var/log/syslog
check that the configuration is correct and reload or restart the server to make sure it's running with the right configuration:
```
# /etc/init.d/dnsmasq restart
```

dnsmasq:

By default: works as a DNS server that serves the data in /etc/hosts.

By default: uses /etc/resolv.conf to find addresses of other DNS to use when a name is not found in /etc/hosts.

To enable the DHCP server, uncomment:

    dhcp-range=192.168.0.50,192.168.0.150,12h

in /etc/dnsmasq.conf and set it to the range of addresses you want to serve. Pay attention to never put two DHCP servers on the same local network, or they will interfere with each others.

To test if the DHCP server is working, use dhcping (not installed by default on Ubuntu).

To communicate other information like DNS, gateway and netmask to the clients, use this piece of dnsmasq.conf:

    # For reference, the common options are:
    # subnet mask - 1
    # default router - 3
    # DNS server - 6
    # broadcast address - 28
    dhcp-option=1,255.255.255.0
    dhcp-option=3,192.168.0.1
    dhcp-option=6,192.168.0.1
    dhcp-option=28,192.168.0.255

Problems found today:

changing the name of the local machine in /etc/hosts breaks sudo, and without sudo it's impossible to edit the file. The only way to fix this is a reboot in recovery mode.
dhclient -n -w is different than dhclient -nw

Quick start examples with tar:

    # Create an archive
    tar zcvf nmap.tar.gz *.deb

    # Extract an archive
    tar zxvf nmap.tar.gz

    # Look at the contents of an archive
    tar ztvf nmap.tar.gz

Quick & dirty way to send a file between two computers without web server, e-mail, shared disk space or any other infrastructure:

    # To send
    nc -l -p 12345 -q 1 < nmap.tar.gz

    # To receive
    nc 10.5.15.123 12345 > nmap.tar.gz

    # To repeat the send command 20 times
    for i in `seq 1 20`; do nc -l -p 12345 -q 1 < nmap.tar.gz ; done

Update: Javier Fernandez-Sanguino writes:

Your "XXX day in Addis" is certainly good reading, nice to see somebody reviewing common tools from a novice point of view. Some comments:

Regarding your comments on how to troubleshoot network connectivity problems I just wanted to point you to the network test script I wrote and submited to the debian-goodies package ages ago. It's available at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=307694 and should do automatically most of the stuff you commented on your blog.

Your example to test hosts alive in the network using nmap -sP 10.5.15.0/24 is good. However, newer (v4) versions can do ARP ping in the local network which is much more efficient (some systems might block ICMP outbount), that's the -PR option and should be enabled (by default). See http://www.insecure.org/nmap/man/man-host-discovery.html Also, you might want to add a '-n' there so that nmap does not try to do DNS resolution of the hosts (which might take up some time if your DNS does not include local IPs)

tcpdump, it would be wiser to turn novice users to ethereal since it has a much better UI than tcpdump and it is able to dissect (interpret) protocols that tcpdump can't analyse.

you are missing arp as a tool in itself, it is useful to debug network issues since if the host is local and does not show up in arp output either a) it's down or b) you don't have proper network connectivity. (If you are missing an ARP entry for your default gateway your setup is broken)

Update: Marius Gedminas writes:

Re: http://www.enricozini.org/blog/eng/third-day-in-addis

In my experience if sudo cannot resolve the hostname (e.g. if you break /etc/hosts), you can still use sudo, but you have to wait something like 30 seconds until the DNS request times out.

I tried to break my /etc/hosts (while keeping a root shell so I can fix it if something goes wrong), but couldn't even get the timeout now. Sudo just said unable to lookup $hostname via gethostbyname() and gave me a root shell.

2009-06-06

eng etiopia pdo tips

Archive of all posts