Category: Linux

Linux and the Open Source Lifestyle

Let’s Build a Server: Part 2 – Monitoring

by

Monit

Last time, in Part 1, we discussed setting up a firewall and an email relay so notifications from the firewall could get to us.

Now, in Part 2, we’re going to talk about more signal. Server monitoring and alerting. Our primary software for monitoring is Monit.

Monit has a single configuration file, but many distributions, including mine, set up a /etc/monit.d folder so you can divide your monit configuration into different files.

Once it is running, you can monitor its status by running
monit status
It will show the status of whatever is monitoring. There is also an optional web component, if you want to check status in a web browser.

What can you monitor?

Monit can monitor any program and restart it if it crashes.
check process nginx with pidfile /var/run/nginx.pid
start program = "/bin/systemctl start nginx.service"
stop program = "/bin/systemctl stop nginx.service"
if failed host 127.0.0.1 port 80
protocol http then restart
if 5 restarts within 5 cycles then timeout

As you can see, the simple scripting language allows you to not only restart, execute programs, but alert the user.

Not only can it make sure something is running, but it can monitor its resource usage, as well as system resource usage. It can monitor processes, network connections, programs and scripts, files, directories, etc.

An Alternative to Email Alerts

The default for an alert is to send an alert email, but for bigger emergencies, a phone push notification is also useful.

Monit provides a simple instruction on how to set it up for Pushover. There is also the alternative of PushBullet.

Pushover costs $5 per platform(Android, iOS, Desktop) to use on as many devices as you want. There is a per application limit of 7,500 messages per month. Pushbullet is, by comparison, free. The basic difference as I see it is that Pushbullet is more geared toward the consumer, and Pushover is more geared toward developers in how it was initially set up. They do have similar feature sets though.

Here is Monit’s suggested Pushover script, which can be run instead of an email alert.

/usr/bin/curl -s
-F "token=your_mmonit_or_monit_app_token"
-F "user=your_pushover_net_user_token"
-F "message=[$MONIT_HOST] $MONIT_SERVICE - $MONIT_DESCRIPTION"
https://api.pushover.net/1/messages.json

Here is an alternative version for Pushbullet

curl -u <your_access_token_here>: -X POST https://api.pushbullet.com/v2/pushes --header 'Content-Type: application/json' --data-binary '{"type": "note", "title": "$MONIT_HOST", "body": "$MONIT_SERVICE - $MONIT_DESCRIPTION"}'

Conclusion

In all cases, monit allows you to monitor your system and take action based on a change in performance. The complexity of your rules is entirely up to you. But, if you give thought to their setup, you can not only be told when there is a server emergency, but the system can take action to fix it.

Let’s Build a Server: Part 1 – The Firewall

by

Tux, the Linux penguin

Necessity is the mother of invention. It is once again time to upgrade the Gadget Wisdom servers. And, as I have committed to writing more here, I will be writing some articles on server construction.

Now, this will all be done using a Virtual Private Server, so the hardware is outside of the scope of this series.

The first piece of software I usually install on network accessible servers is the ConfigServer Security & Firewall(CSF). This is a firewall with login/intrusion detection, and security. Most distributions of Linux come with some sort of firewall, but this set of scripts works with iptables to be much more secure.

CSF provides scripting for a firewall, and handles login failure handling for a variety of stock services, as well as unsupported services using regular expressions.

There are a lot of options in the CSF configuration file…read through the description of each…decide which ports you want open, and deploy. CSF will automatically update itself when there is a new version.

In order to ensure notifications from the firewall and other administrative notifications are read, you will likely wish to arrange for the ability to send mail. However, you may not need or wish the trouble of setting up a mail server. The simpler solution is to set up an SMTP relay.

The example below configures Postfix, available with many Linux distributions, for use with a gmail account. Add the following lines to the bottom of your /etc/postfix/main.cf

smtp_use_tls=yes
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_sasl_tls_security_options = noanonymous

Create a file with your gmail credentials.

smtp.gmail.com user@gmail.com:PASSWORD

Then secure the file.

chmod 640 /etc/postfix/sasl_passwd*
postmap /etc/postfix/sasl_passwd

Now, any external email will route through your gmail account. We have now protected our server from a variety of attacks, and ensured, if there is a problem, we’ll be notified of it.

There are alternatives to Gmail. For example, Mandrill offers 12,000 emails per month for free, and 20 cents per thousand after that, and Sendgrid offers 200 emails, and 10 cents per thousand.

You can use Mandrill or Sendgrid instead of Gmail by using the below credentials.

[smtp.mandrillapp.com]:587 USERNAME:API_KEY
[smtp.sendgrid.net]:587 USERNAME:PASSWORD

 

 

Related articles

Revisiting the Chromebook Challenge

by

English: An Acer Chromebook showing the introd...

Back in January of 2013, I took a Chromebook challenge, and invited several other members of the Gadget Wisdom family to try the thing out.

This was the $199 Acer C7 Chromebook. This set up Chromebooks as the successor to the old netbook market…the small portable device you took along for productivity, but was not your primary driver.

At the end of that, I concluded it was a solid investment.

So, has anything changed in nearly two years? More manufacturers are making Chromebooks. The gambit runs from the cheaper ones to the ultra-expensive top of the line Pixel, with regrettably, few options midrange.

While the apps aren’t there, Chrome OS does support native apps, and Android apps are starting to come over. With time, the ecosystem will continue to mature. More on this to come.

 

Related articles

Nginx FastCGI Caching

by
English: Nginx Logo Español: Logo de Nginx
English: Nginx Logo Español: Logo de Nginx (Photo credit: Wikipedia)

Over the last few months, I’ve been doing a lot of work trying to speed up the sites on my server….perhaps to the detriment of this site, Gadget Wisdom.

Gadget Wisdom runs on WordPress on a Nginx web server. To run PHP on an Nginx server, you need to pass requests to a FastCGI server.

Nginx supports caching the responses. So, WordPress generates a page dynamically, Nginx caches the response and can serve the cached version on request. Since the resource intensive part is the application, and most people don’t need a changing page, it works for the majority of issues.

For the last few years, refreshing the cache has been done by sending a request with a specific header. This has the effect of telling the system to generate the page again and store the result. A recent upgrade added in the optional Nginx Cache Purge module. This allows a purge of a specific page using a simple URL scheme.

The net difference between the two in effect is that the purge function removes the cached version to be regenerated on the next load. The header option generates a new version of the page which is stored in the cache. The disadvantage of the Purge module is you have to custom-compile Nginx…which means you have to manually keep up on security bugfixes.

Either way, once you decide on methodology, you also have cache validity. For example, many people opt for a microcache solution…where the cache time is very short, measured in seconds. This means that only when the site is being hit will people be served ‘stale’ pages.

The alternative is a very long cache time…measured in hours/days. As long as you have a cache refresh function available, such as the options mentioned above so you can remove the stale pages on demand, you can keep the pages around for longer periods of time.

Right now, my cache validity time continues to rise over time. You also have browser caching. Right now, images are instructed to be cached by your browser for days. I don’t usually change my images much after posting…or at all.

So, this post hopefully covered the basic decision making process for FastCGI caching on Nginx. In Part 2(if I get to it), we’ll cover some of the settings to allow this, as well as some of the considerations you have to make while coding this.

Trimming Your SSD

by

Kingston SSD Ready for InstallationNearly three years ago, I wrote an article on optimizing for SSDs under Linux. Recently, I decided to revisit the issue after reading a recent blog post.

The recommendation at the time was to enable TRIM support, using the discard option to mount the drive. The first question is, if this is such a good idea, why isn’t it enabled by default? Why do you have to add it to your options, like below?

/dev/sda1 / ext4 discard,defaults

It turns out that enabling the discard option does have a performance hit on deletes. So, how do you keep your SSD Trimmed and avoid a costly performance penalty?

It turns out you can trim manually using the fstrim command, and set up a cron job to run this command once a day. The command takes only one argument by default, the mountpoint of the partition.

Seems like something worth thinking about. However, with the majority of the systems I run SSDs on, the solid state acts an an OS drive. Therefore, the number of deletes are minimal compared to writes.

In the end, enabling TRIM on your drive ensures that the drive will have the best wear-leveling and performance, but there is a cost. For some systems, it is just easier to mount with the discard option, others to run fstrim.

Related articles

Information Overload – Trying to Reorganize A Workflow

by

In January of 2011, I wrote a story on the subject of Organizing Your Workflow with Instapaper and Pinboard. This was in response to the announcement of the impending closure of Xmarks(which later did not close), and the announcement that Delicious was shutting down. This had brought me to Pinboard.

Pinboard-Home

Pinboard is currently available at a rate of just over $10 for a lifetime subscription, plus $25/yr for an Archival Account.

At the time, I used Instapaper, a Read It Later service, as a holding pen for stories, which I later archived in Pinboard. In April of 2011, I announced the move to Read It Later(now Pocket). There were many good reasons for this, however, the refresh from Read It Later to Pocket made service lean more toward the visual.

Which brings me to May of 2012, where I once again pondered the subject, right after I read Clay Johnson’s book, the Information Diet. At the time, I vowed to get my information overload under control.

Here we are, March of 2013, and…it is worse. I finally declared bankruptcy on Pocket(Formerly Read it Later). I exported everything I was most definitely NOT Reading Later, and sent it to Pinboard. There is some duplication there that has to be cleaned up, but now I have 25,000 bookmarks to go through and prune. The archive of which takes up 25GB.

I’ve come to the conclusion that this isn’t working, but I’m changing plans once again. I need a plan that allows me to reference old material I have in the archive, while keeping track of more relevant material. For now, I’ll be living in Pinboard, without benefit of a secondary service. But I am open to suggestions.

Will update you as this develops.

 

 

Related articles

Running Personal Services on a Low End VPS

by

For those of us who like to tinker with client/server software for personal or household, there are many good options. You can use a Raspberry Pi as a server, for example. You can use an old computer.

Both of these would have services running out of your home or business. But, as we are an increasingly mobile society, you might not have good upstream bandwidth, or your ISP may block ports into your home. So, that is where a low-end VPS offering comes in.

ChicagoVPS.netWe chose ChicagoVPS, which offers a $12/year 128mb VPS, with 10GB of storage space and 100GB of monthly bandwidth. That is more than enough for personal use. They offer three locations: Chicago, Buffalo, or LA. There are similar services averaging around $12-15 a year.

This is not the sort of service where you expect a lot of reliability. The service has had some hiccups,  but as long as you backup and take adequate steps you should on any service, there shouldn’t be any problem.

On a 128mb instance, I have Tiny Tiny RSS running, as well as ZNC, and a few other random services that I only use for my own personal interests.

What do you think? Do you have any other recommendations for a tiny VPS? Do you have alternative providers you recommend for cheap VPS services?

Related articles

Reader Refugees – The Death of Google Reader

by

English: Screenshot application of google read...

It has been a long two weeks since Google announced the death of Google Reader. This left many people scrambling for new solutions as the clock countdowns to its shutdown on July 1st, 2013.

 

There are many alternatives out there of various types. Feedly, for example, has been working on a Reader alternative that uses the same API. The service is also working to offer more Reader like features to welcome the over half a million Reader Refugees. They seem very determined to be the new Reader, and are even welcoming those interested in their API-compatibility to enable their applications to keep working.

 

For me, however, this was too fancy. Most of us who are interested in replacing Reader emphasize text. We want the experience of a newspaper, not a magazine. Most Readers use the traditional Inbox style of receiving, akin to email programs. The information is the most important part.

 

For this, I skipped over TheOldReader, which was designed to mimic the original Google Reader design, and went straight to two open source projects.

 

Newsblur

newsblur

 

 

 

 

Newsblur is the brainchild of Samuel Clay. In addition to the standard Inbox display, it allows you to view the original site in context, or the feed version. It also offers options for sharing a feed of what you find most interesting with others, and teaching the application what you find interesting, so it will highlight that. Newsblur offers a public API for people to build on, and the entire codebase is open source.

 

There is an Android app, and Clay is looking for an Android developer, but complaints of crashes, as it is an open-source project, were quickly cleaned up by a volunteer. There is also an iOS app.

 

The exciting thing is that with the renewed interest, Clay is ramping up. A new host for the service, more robust infrastructure, and more.

 

Newsblur can be installed by you as a standalone product, or you can pay for their hosted service, currently at $24/year.

 

Tiny Tiny RSS

tt-rss

Tiny Tiny RSS(TT-RSS) is another open-source project. Like Newsblur, it offers an API, a web interface, and an Android app. There are no hosted options for this, so you have to roll your own, which is what I did, using a Low-End VPS.

It can run on simple hardware, out of your home or on rented space.

Conclusion

 

The truth is, if you host your own solution, you can be reasonably sure it will continue to be there(as long as you keep paying the bills).

But there is something to be said for having someone else worry about it, as well as supporting the developer. So, even though I’ve settled into TT-RSS(and gave the developer the $2 for the Android app), I paid for a year of Newsblur so I could see how it develops. I never considered self-hosting of a Newsblur instance.

Next, I’ll spend a little time on where I’m hosting TT-RSS and why.

 

 

 

Related articles

Some Changes to the Site

by

Loading Mail onto Railway Post Office Car

It is time for some housecleaning.

  • There is a new layout to the site. Feel free to provide comment on this post of what you think
  • We have a weekly newsletter, which will send you the posts if you prefer that to other means of subscription. This replaces the previous Feedburner Email service with Mailchimp.
  • We’ll be doing some cleanup of the site content to make it easier to navigate and trying to launch some new content.
  • Feedburner feeds are no longer active, and will redirect to the native theme.
  • We can also be engaged on Twitter or Facebook, should you wish it.

[newsletter-sign-up-form].