Posts Tagged PCL

Syslogd: the logging daemon

BitacoraWhen a service fails, it reports what happened or when something goes wrong in your machine (for instance, an hardware issue) is also reported. Those messages can be found in the /var/log directory.  For example, many messages are reported to /var/log/syslog or /var/log/messages file.  On the other hand, if a service generates a lot of messages it’s most likely they will be written in a separated file like mail servers or Apache do.

All this is done by a daemon called syslogd, which is a system utility  providing  support for message logging on *nix systems. But logging is not reserved for system applications, you can also log your messages using syslogd, setting a proper rule in the /etc/syslog.conf file. So, let’s see how to do it…

Configuration file (/etc/syslogd.conf or /etc/rsyslogd.conf)

In the /etc/syslog.conf file (ore /etc/rsyslog.conf) are defined the rules for logging with syslogd, a rule per line. Every rule follows this syntax:

selector action

The selector part is made up of facility.priority, so the complete syntax would be:

facility.priority action

The facility is one of the following keywords:

Facility Description
auth Security/authorization messages
authpriv Security/authorization messages (private)
cron Clock daemon (cron and at)
daemon System daemons without separate facility value
kern Kernel messages
lpr Line printer subsystem
mail Mail subsystem
mark For internal use. Don’t use when making rules
news USENET news subsystem
security  (same  as auth) Obsolete, use auth instead
syslog Messages generated internally by syslogd
user Generic user-level messages
uucp UUCP subsystem
local0 through local7 Reserved for local use

The priority can be one of keywords listed on the following table. Al messages will be reported by priority, in ascendant order. For example, if you specify alert priority it will report alert, emer y panic priorities, but it will not report  from crit, to debug priorities.

Priority Description
debug Used to debug services, for instance if they are not working properly
info Used to report informative messages
notice Like info priority, but making notice something  that can be relevant
warning Used to report warnings. It can give you some clues about errors (if any)  or just show you if there’s something  not working as expected, but it  it can be working anyway
warn Same  as warning
err Used to report errors. For instance, if you have a misconfiguration in a service, it will report those errors
error Same as err
crit Used to report more critical errors. For example, hardware errors
alert Used to report even more critical errors. Action must be taken immediately. For instance, database corruption.
emerg Used to report really critical errors. Most likely the service is unusable
panic Same as emerg
none Used to disable a facility’s report.

The action field describes what to do with the reported message. Commonly all messages are written to a file know as logfile,  but there are also other actions, like forwarding messages to another host. So, the action field can be one of the following:

Action Description
/path/to/logfle Write messages to a logfile
| fifo Use a fifo or named pipe as a destination for log messages. This is handy for debugging or sending mails. Note that the fifo must be created with the mkfifo(1) command before syslogd(8) is started
/dev/tty[1-6] Write messages in /dev/tty[1-6] console. Note, /dev/console will work as well
@192.168.0.1 Forward messages to host 192.168.0.1 via UDP.  Due to the nature of UDP, you will probably lose some messages in transit.  If you expect high traffic volume, you can expect to lose a quite noticeable number of messages. Note: in order to accept messages, the remote server must run syslogd with the -r option (on Debian this option can be given in the /etc/default/syslogd file or o /etc/default/rsyslog)
:omrelp:192.168.0.1:2514 If you would like to prevent message loss, use RELP
lgallard, atorres List of user. By default, critical messages are sent to root

Modifiers

Basically there are three modifiers: =,  ! and *.  The “=” modifier makes syslogd to report only messages with the exact priority . For Instance:

mail.=error                       /var/log/mail.error

Here syslogd will report only error messages. Without the = modifier it should report error, crit, alert and panic messages. This modifier can only be used with priorities.

The second modifier is the “!”, which inverts the meaning of the rule. For example:

mail.!error                       /var/log/mail.error

Syslogd will report messages with less than error priority, ergo warning, notice, info and debug. If you want to exclude just one priority, you must use the !=  combination.

Finally, the “*” modifier lets you select among different facilities or services. For instance:

mail.*                       /var/log/mail.log

Here, all messages from mail facility will be saved to /var/log/mail.log file, not matter the priority. Another example:

*.info                       /var/log/info.log

It doesn’t matter what facility, all messages with info priority will be saved to /var/log/info.log file.

Semicolon and colon operators

The semicolon operator lets you write several rules in a more compact way. For example:

mail.=info                       /var/log/info.log
mail.=notice                     /var/log/info.log
auth.=info                       /var/log/info.log

The above rules can be written in one line:

mail.=info;mail.=notice;auth.=info   /var/log/info.log

On the other hand, if you want select several facilities, you can use the colon operator. For instance:

mail.info                       /var/log/info.log
auth.info                       /var/log/info.log

You can write the above rules in one line, like this:

mail,auth.info                  /var/log/info.log

The big difference between the semicolon and the colon operator is that the latter only separates facilities and the first one can separate facilities and priorities, even if they are not compatible.

Logging synchronously

Some logs must be monitored in real time, for instance when debugging a service. The thing is that syslogd writes messages when its buffer is full, i.e., asynchronously . If you want to write messages synchronously put a “-” before the log’s file path.

Examples

Here’s some examples I made up, and others taken from /etc/syslog.conf:

local3.info                     /var/log/mylog

Use local3 facility, reporting info messages to /var/log/mylog file.

auth,authpriv.*                 /var/log/auth.log

Report all priorities for auth and authpriv facilities to /var/log/auth.log file.

mail.warn                       -/var/log/mail.warn

Warn messages from mail will be saved into /var/log/mail.warn file synchronously.

mail.!=error                       /var/log/mail.error

All except error messages will be saved into /var/log/mail.error file.

*.=debug;\
	auth,authpriv.none;\
	news.none;mail.none	-/var/log/debug

All debug messages will be saved to /var/log/debug synchronously, except those from auth, authpriv, news and mail facilities.

The logger command

If you want to communicate to syslogd, you can use the logger command.  For instance, to send a message with info priority and local3 facility, just type:

logger -p local3.info “This is a message”

References

,

5 Comments

Active FTP vs. Passive FTP

FTP is a TCP service that uses a data port and a control or command port. Commonly those ports are 20 and 21, respectively. But it’s not always that way…specially if you are behind a firewall.

Active FTP

Let’s see how ftp connections works normally:

  • The client connects from a random unprivileged port (N >1024) to ftp server’s command port (21)
  • The client starts listening on port N+1 and sends this ftp command port to the server
  • The server will connect back to port N+1 from its data port (20)
activeftp

Active FTP

Here is a connection example:

  1. Client’ s command port (1026) connects to server’s command port (21) and sends data port 1027
  2. Server sends back an ACK to the client’s command port
  3. Server starts a connection between its data port (20) and client’s data port (1027)
  4. Client sends an ACK to server

Passive FTP

The active mode has a problem, if the client is behind a firewall the server won’t be able to establish a connection to the client’s random data port (the one which is above 1024). In order to fix this problem, there’s another connection method called passive FTP:

  • The client opens two random unprivileged ports (N > 1024 and N+1)
  • The first port (command port) connects to port 21 on server side (command port) and sends a PASV command
  • Then, the server opens a random unprivileged port (P > 1024) and sends the PORT P command back to the client
  • Finally, the client starts a connection from port N+1 to port P on the server to transfer data
passiveftp

Passive FTP

Here’ s an example.

  1. Client’ s command port (1026) connects to server’s command port (21) and sends PASV command
  2. Server replies to the client with port 2024, the random data port opened for data connections
  3. The client starts a connection from its data port (1027) to server’s data port (2024)
  4. Finally, the server sends back an ACK  to the client’s data port

Reference

,

2 Comments

RAID

raid-hard-drive-arrayAccording to Wikipedia: “RAID is now used as an umbrella term for computer data storage schemes that can divide and replicate data among multiple hard disk drives. The different schemes/architectures are named by the word RAID followed by a number, as in RAID 0, RAID 1, etc. RAID’s various designs all involve two key design goals: increased data reliability or increased input/output performance. When multiple physical disks are set up to use RAID technology, they are said to be in a RAID array. This array distributes data across multiple disks, but the array is seen by the computer user and operating system as one single disk. RAID can be set up to serve several different purposes”.

The distribution of data across multiple drives can be managed either by dedicated hardware or by software, or even by hybrid RAID based on software ad specific hardware.

According to the RAID you choose, these are some RAID’ s features:

  • Data reliability
  • Better fault tolerance
  • Better performance
  • More space
  • Integration of low cost disks

Original RAID’s  specification suggested some numbers called “RAID levels”, each of them with theoretical pros and cons. Over the years, different RAID concepts have appeared, but most of them differs substantially from the original RAID levels, but it’s still common to use a number (e.g, RAID7)
Read the rest of this entry »

,

2 Comments

Working with tar

file-roller

If you’ve never worked with tar before or if you know few options this post will show how to use tar, at least the basic options. First of all, let’s see an extract from tar’s manual description:

“Tar stores and extracts files from a tape or disk archive. The first argument should be a function; either one of the letters Acdrtux, or one of the long function names.. A function letter need not be prefixed with ‘-’, and may be combined with other single-letter options. Some options take a parameter; with the single-letter form these must be given as separate arguments. A long function name must be prefixed with –”

Ok, let’s see some of those functions and options…
Read the rest of this entry »

,

1 Comment

The netstat command

Comando netstat

The netstat command

According to Wikipedia, “Netstat (network statistics) is a command-line tool that displays network connections (both incoming and outgoing), routing tables, and a number of network interface statistics”.

By default, netstat without any option shows current connection status by listing connectors. But let’s see other options you can use with the netstat command…

Options

Option Description
-e Display additional information. Use this option twice for maximum detail.
-r Displays the contents of the IP routing table (like route command).
-i Displays network interfaces and their statistics.
-n Addresses, port numbers, users are expressed numerically and no attempt is made to determine names.
-l Show only listening sockets.
-a Show both listening and non-listening sockets (displays all active TCP connections and the TCP and UDP ports on which the computer is listening).
-s Display summary statistics for each protocol. The -p parameter can be used to specify a set of protocols.
-t Show only tcp connections.
-u Show only ucp connections.
-c This will cause netstat to print the selected information every second continuously.
-p Show the PID and name of the program to which each socket belongs.

Read the rest of this entry »

,

No Comments

Understanding the free command

free

free command's output and System Monitor

Free displays the amount of free and used memory in the system, according to the manual (man free). But here’s one of the confusing terms in Linux: “used memory”. From others operating systems, we are used to the idea that “used memory” reefers to memory we already used, so we can’t use it in the future. But in Linux, “used memory” stands for memory ẗaken by the system, so it can be used by users or applications. From this point of view in Linux the bigger the used memory, the better. If you have 1 GB RAM and used memory of 900 MB, please don’t buy another extra 1 GB ram, because the more likely you will have 1.9 GB of used memory…again, don’t worry it’s normal.

Read the rest of this entry »

,

No Comments

Review questions for PCL I & II exam (Profesional Certificado Linux)

logo-iseit

If you taking the Profesional Certificado Linux (PCL)  curse at ISEIT institute, here’ s a few review questions for the certification exams PCL I y PCL II (in Spanish)

The answers are up to you :)

Institute’s URL : http://ve.iseit.net

,

2 Comments

Vi Quick reference

vicmd

Tire of those non-fancy text editors? You are tied up to a specific editor? Your coworkers make laugh of you because you don’ t understand vi?…Don’t worry anymore!! Here’s a quickguide for vi.

How Vi works?

Maybe you have got tired of vi because anytime you try to use it, you don’t get the results you want. Ok, let’s talk about how vi works. It has three operation modes:

  • Command mode: In this mode, vi expects an action to be performed, like copying, pasting or just moving the cursor. This is its default behavior, so when you open a file and start writing you won’t see what you are writing because you are performing commands.
  • Inserting mode: In this mode you can in deed write into the file, but before you have to use one of the inserting command like “i”.
  • Command extended mode: This is a special mode where you can pass arguments to commands. In order to enter to this mode just write a colon and a command. For instance:

:1,4 y

It means to copy from line 1 to 4. Now lets see some commands grouped according to their function:

Cursor movement

h Move left
j Move down
k Move up
l Move right
w Go to start of word (next word)
e Go to end of current word
b Go backward by word
0 Star of line
$ End of line
nG Go to line n. G alone means got to file’s last line
:n Go to line n

Read the rest of this entry »

,

No Comments

Terminal’s shortcuts

Fingerprint Evidence

How do you clear the screen without typing the clear command? my students always ask about this little trick I do while working on a terminal. Well, the answer is by using this shortcut: Ctrl+l. Immediately after they ask me if there are any other shortcuts, and I reply “Yes, of course…but I don’t remember them. When I start using Linux I read some of them, but I only remember the Ctrl+l shorcut”.

So, I decided it was the time to recheck those shortcuts and post them here…

Terminal’s shortcuts

This table shows some shortcuts. Keep in mind some can work o not depending on the console you use. For instance, Alt+f is a defined shortcut in Gnome’s terminal.

Shortcut Description
Ctrl+b Move back one character.
Ctrl+f Move forward one character.
Ctrl+d Delete the character underneath the cursor.
Ctrl+_ or Ctrl+x Ctrl+u Undo the last editing command. You can undo all the way back to an empty line.
Alt+u upcase-word
Alt+l downcase-word
Altc+c capitalize-word
Movement Commands
Ctrl+a Move to the start of the line.
Ctrl+e Move to the end of the line.
Alt+f Move forward a word, where a word is composed of letters and digits.
Alt+b Move backward a word.
Ctrl+l Clear the screen, reprinting the current line at the top.
Cut and paste commands
Ctrl+k Cut the text from the current cursor position to the end of the line, and copy it to the buffer. To cut text is also know as ” killing”. The text killed is saved in a kill-ring for later use by pasting (yanking).

 

Alt+d Kill from the cursor to the end of the current word, or, if between words, to the end of the next word. Word boundaries are the same as those used by Alt+f.
Alt+DEL Kill from the cursor to the start of the current word, or, if between words, to the start of the previous word. Word boundaries are the same as those used by Alt+b.
Ctrl+w Kill from the cursor to the previous whitespace. This is different than Alt-DEL because the word boundaries differ.
Ctrl+y Yank the most recently killed text back into the buffer at the cursor.
Alt+y Rotate the kill-ring, and yank the new top. You can only do this if the prior command is Ctrl+y or Alt-y.
Commands in the History
Ctrl+r Search backward in the history for a particular string.
Ctrl+s Search forward through the history (it might work or not)
Alt+< Move to the first line in the history.
Alt+> Move to the end of the input history, i.e., the line currently being entered.

Shortcuts and arguments

You can pass numeric arguments to terminal commands. For example, to give the Ctrl+d command an argument of 10, you could type

`Alt+1 0 Ctrl+d’, which will delete the next ten characters on the input line.

Custom shortcuts

You can go forward and define your own shortcuts in bash. To do so just add the shortcut you want on your ~/.inputrc file (or /etc/inputrc for all users). For instance, the search forward function didn’t work on my computer so I had to put this one:

“C-h”:history-search-forward

Here “C-h” means Ctrl+h. You can even define a shortcut for a command like ‘ps aux‘ adding this line:

“C-o”:’ps axun’

In order to this changes take effect  the inputrc file have to be reread using this shortcut Ctr+x Ctrl+r.

References

,

No Comments

The find command

lupa.jpg

On Linux we can perform file searching in many ways. For Instance, from Nautilus (the Gnome’s file manager) we can search on the current directory for all mp3′s files using this expression ‘*.mp3′. But if we need to do more complicated searches, for example all mp3′s files greater than 5 MB, Nautilus is not the answer. In this cases we have to use a versatile and powerful command: find.

Using find

Find lets us make file searching on the system according to an expression. The basic syntax is:

find path expression

Where expression is made up of options (change general behavior) , tests (to return true or false) and actions (for processing the files found)

Lets see some options, tests and actions…

Options

-maxdepth n Descend  at  most  levels
-mount Don’t  descend  directories on other filesystems

Tests

n for exactly n
+n for greater than n
-n For less than n
-amin n File was last accessed n minutes ago
-anewer file File  was last accessed more recently than file was modified
-atime n File was last accessed n*24 hours ago
-cmin n File’s status was last changed n minutes ago
-cnewer file File’s status was last changed more recently than file was modified.
-ctime n File’s  status  was  last  changed  n*24 hours ago
-mmin n File’s data was last modified n minutes ago
-newer file File  was  modified more recently than file
-mtime n File’s data was last modified n*24 hours ago
-empty File is empty and is either a regular file or a directory
-executables Matches files which are executable and directories which are searchable
-readable Matches files which are readable
-fstype type File is on a filesystem of type type
-uid n File’s numeric user ID is n
-user user_name File is owned by user user_name
-gid n File’s numeric group ID is n
-group group_name File belongs to group group_name
-name pattern Base  of  file  name  (the  path  with the leading directories removed) matches shell pattern pattern
-nogroup No group corresponds to file’s numeric group ID
-nouser No user corresponds to file’s numeric user ID
-perm mode File’s permission bits are exactly mode (octal or symbolic)
-perm -mode All  of  the permission bits mode are set for the file
-perm /mode Any  of the permission bits mode are set for the file
-size n[cwbkMG] File uses n units of space
-type c File is of type c (block, character, directory, symbolic link, regular file, etc.

Actions

-delete Delete  files;  true  if removal succeeded
-exec comando ; Execute command; true if 0 status is returned
-exec comando ‘{}’ \; This  variant of the -exec action runs the specified command on the selected files, but the command line is built by appending each selected file name at the end

Examples

find /home -name ‘*.bash*’ Any file which name contains .bash in its name, on the /home directory
find $HOME -mtime 0 Files on user’s home directory modified less than 24 hours ago
find -perm 644 Search for files which have read and write permission for their owner, and group, but which other users  can  read but not write to.  Files which meet these criteria but have other permissions bits set (for example if someone can execute the file) will not be matched
find -perm -644 Search for files which have read and write permission for their owner and group, and which other users  can  read, without  regard  to the presence of any extra permission bits (for example the executable bit).  This will match a file which has mode 0777, for example.
find -perm /222 Search for files which are writable by somebody (their owner, or their group, or anybody else)
find -perm /u=w Same than above example
find /home -nouser -exec rm -r ‘{ }’ \; Delete files on /home directory that belongs to nobody
find /home -size +1G Search for files greater than a 1 GB on /home
find / -size +1G -user lgallard Search for files greater than a 1 GB on /home which belongs to lgallard

References

,

2 Comments