AppBeat Blog

Perfect Monitoring for Your Cloud

New scheduling algorithm

In last few days we upgraded our core scheduling algorithm which assigns agent resources (CPU, network) to your monitoring tasks.

New version now handles agent resources much better and distributes monitoring tasks more equally. This results in much lower average resource usage on our entire pool of monitoring agents.

Here is screenshot of CPU usage on one of our agents where this effect is clearly visible:

Testing AppBeat monitoring automation

We are currently finishing command line tool for basic AppBeat management. Soon you will be able to get your system status, pause / resume / delete services and checks, create new checks.

For example, this is minimum configuration for creating new check:


{
	"Agent": "Web",
	"Name": "Check1",
	"Description": "Check created with AppBeat command line tool / API",
	"CheckIntervalInSeconds": "60",
	"AgentSpecificSettings": {
		"URL": "https://appbeat.io",
		"TIMEOUT": "15",
		"METHOD": "GET",
		"HTTP_VER": "http1.1",
		"IP_VER": "ipv4"
	}
}

AppBeat command line tool will be available for Windows, Linux and Mac.

We simply love LINQ!

It is true! It is so powerful and useful feature that we use it as much as possible.

Here is recent code snippet which uses "Linq magic", resulting in much smaller and more readable code:

Thank you LINQ team!

Fully customizable web monitor

We are currently actively testing advanced web monitor which will allow you to:

  • select HTTP method (GET, POST, PUT, DELETE, HEAD, TRACE)
  • select HTTP protocol version (1.0, 1.1 or newest HTTP 2)
  • select IP version which should be used when sending request to server (IPv4 or IPv6)
  • specify custom headers (you will be able to set custom user agent, ...)
  • define custom payload (for example for JSON POST, ...)

Stay tuned!

AppBeat web app can now run on Linux!

We have successfully migrated our web application from .NET Core RC1 to .NET Core RC2 and published it on test Linux instance (CentOS). Best of all: AppBeat binaries are 100% identical to those on Windows (same cross platform codebase!) and were simply copied to Linux... And it works! It works great!

Here are some setup details for those who are interested.

First, you have to install .NET Core runtime by following simple instructions on Microsoft .NET Core website.

ASP.NET Core web applications are running on lightweight web server called Kestrel, but since it lacks some of important web server features (for example HTTPS, various built-in tools, ...) it is usually configured to run in tandem with other web server of your choice. On Windows this is IIS, on CentOS we selected NGINX. Those web servers then forward your request to Kestrel which runs your ASP.NET Core web application, creates output and returns it back to main web server.

To setup NGINX to forward requests to Kestrel you have to edit nginx.conf with following (adapt if needed):

location / 
{
proxy_pass http://127.0.0.1:5000;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Original-Proto $scheme;
proxy_set_header X-Original-For $remote_addr;
proxy_http_version 1.1;
}

Then we installed supervisor to monitor our Kestrel process and make sure it is always running:
sudo yum install supervisor

We created kestrel_default.ini file in /etc/supervisord.d/kestrel_default.ini with following content:

[program:kestrel_default]
command=dotnet /usr/share/nginx/WebApp/AppBeat.Web.dll
directory=/usr/share/nginx/WebApp
autorestart=true
autostart=true
stdout_logfile=/var/log/AppBeat/app_std_out.log
stderr_logfile=/var/log/AppBeat/app_err.log

After you saved file you can start Kestrel process by typing:

sudo supervisorctl start kestrel_default

Make sure nginx and supervisor are started on system boot:

sudo systemctl enable nginx
sudo systemctl enable supervisord

This is pretty much it :) We are now prepared for future! If you have questions you can reach us on @AppBeatMonitor

How to develop PagerDuty support in C#

We are currently working on version of AppBeat which will have support for PagerDuty (third party incident management tool).

Their API is very easy to use. Here is code example from AppBeat which sends PagerDuty notifications:

namespace AppBeat.Core.Notifications
{
    class PagerDuty
    {
        //NOTE: this is dispatched in separate thread, exception handling is done by NotificationManager
        public static bool Send(ThirdPartyDispatcher.CustomNotification notification)
        {
            var settings = (Types.CustomNotificationPagerDuty)notification.Settings;

            if (string.IsNullOrWhiteSpace(settings.ServiceKey))
            {
                throw new Exception("ServiceKey is required!");
            }

            string incidentKey = Guid.NewGuid().Normalize();

            var req = new PagerDutyReq()
            {
                service_key = settings.ServiceKey,
                description = notification.ShortMessage,
                incident_key = incidentKey,
                details = new { Message = notification?.Details?.Output?.Details },
                event_type = "trigger",
                client = "AppBeat",
                client_url = "https://appbeat.io"
            };

            using (var wc = new WebClient())
            {
                var res = JsonConvert.DeserializeObject<PagerDutyRes>(wc.UploadString("https://events.pagerduty.com/generic/2010-04-15/create_event.json", JsonConvert.SerializeObject(req)));
                return res.IsOk();
            }
        }

        class PagerDutyReq
        {
            public string service_key
            {
                get; set;
            }

            public string event_type
            {
                get; set;
            }

            public string incident_key
            {
                get; set;
            }

            public string description
            {
                get; set;
            }

            public string client
            {
                get; set;
            }

            public string client_url
            {
                get; set;
            }

            public object details
            {
                get; set;
            }
        }

        class PagerDutyRes
        {
            public string status
            {
                get; set;
            }

            public string message
            {
                get; set;
            }

            public string incident_key
            {
                get; set;
            }

            public bool IsOk()
            {
                return string.Compare(status, "success", StringComparison.InvariantCultureIgnoreCase) == 0;
            }
        }
    }
}

To use this code you must first create service on PagerDuty with API integration. You then receive unique service key which you use in your API. It can't be more simple than this :)

New website monitoring features (coming soon)

Hello AppBeat users.

Most of our users are currently very happy with our service which gives us a lot of confidence that we are building right tool for you. Thank you for this!

We just wanted to notify you that we are currently finalizing work on new AppBeat version (1.1) which will bring new cool features. We will give you more control about core AppBeat behaviour and we will add more personalization options (we won't say much more at this point, stay tuned for version 1.1 :)

After we publish version 1.1, we already have new very useful feature planned for version 1.2 (it should also stay secret for now).

If you have your own ideas which should be implemented in AppBeat please submit them to our idea pool. Most voted ideas are probably going to be implemented in one of future versions.

AppBeat Wish List - how would you improve our monitoring solution?

What would you like to see in future versions of our monitoring app?

We have created draft of initial ideas. We would like to encourage you to vote for your favorite feature or publish your own request.

Currently we have following ideas:

  • AppBeat monitoring API
    • For example, custom clients can be made.
  • Realtime user monitoring reports
    • Put AppBeat JavaScript snippet on your page and monitor real time user performance.
  • Mobile apps
    • Create mobile apps for Android, Windows Phone for easy management.
  • Exporting reports
    • Excel, CSV, ...
  • Smart agents
    • Monitor CPU and disk usage, ...
  • Improved public status pages
    • For example public status widget integrated directly to website.
  • Support for web hooks
    • When check changes status call user defined URL and pass task details (task name, status, time)
  • More notification channels
    • Currently there is support for email and SMS. Add support for push notifications, publishing twitter status, ...
  • Vulnerability report
    • Create periodic vulnerability report for websites.
  • Periodic full page speed testing
    • Download entire page with all resources (images, scripts, ...). Monitor download time and performance grade.
  • Audio alarm on failure
    • If error occurs in "Live status" view play alarm in web browser.
  • Scheduled and triggered tasks
    • Extend AppBeat monitoring platform with possibility to run scheduled / triggered web tasks (similar to Zapier or IFTTT)

These are initial ideas that we received from some of our users. You are welcome to vote and submit your own ideas on http://appbeat.idea.informer.com.

Coming very soon: Website Speed Test

We are currently doing final internal review of our API for synthetic website speed testing!

var harResponse = AppBeat.PageSpeed.API.RunTest(test.PublicId, test.Url, test.Location);

We have build entire infrastructure for this from grounds up in record time. We will provide multiple test locations and simple page for everyone to use.

Later we will provide special REST API for website speed testing and we will also integrate this functionality into AppBeat monitoring as free bonus.

Stay tuned :)

C# - how to lock without deadlocks

In C# there is built-in keyword lock which enables you to create thread-safe methods. However, you must be aware that if you are not careful, you can create deadlock conditions where your method waits for lock indefinitely.

How lock keyword works? As described in "Locks and exceptions do not mix" blog article, in C# 4 compiler translates lock statement into this:

bool lockWasTaken = false;
var temp = obj;
try { Monitor.Enter(temp, ref lockWasTaken); { body } }
finally { if (lockWasTaken) Monitor.Exit(temp); }

We can use this to create utility method which works similarly to original lock, but waits only specified amount of time and prevents deadlocks:

using System;
using System.Threading;

namespace AppBeat.Utility
{
    public static class Concurrency
    {
        /// <summary>
        /// Method tries to run <seealso cref="doAction"/> in thread-safe manner but waits max <seealso cref="millisecondsTimeout"/> milliseconds for lock. If it can not acquire lock in specified time, exception is thrown.
        /// This method can be used to prevents deadlock scenarios.
        /// </summary>
        public static void LockWithTimeout(object lockObj, Action doAction, int millisecondsTimeout = 15000)
        {
            bool lockWasTaken = false;
            var temp = lockObj;
            try
            {
                Monitor.TryEnter(temp, millisecondsTimeout, ref lockWasTaken);
                if (lockWasTaken)
                {
                    doAction();
                }
                else
                {
                    throw new Exception("Could not get lock");
                }
            }
            finally
            {
                if (lockWasTaken)
                {
                    Monitor.Exit(temp);
                }
            }
        }
    }
}

Instead of this code:

lock (this)
{
    //do something
}

You could then have something like this:

AppBeat.Utility.Concurrency.LockWithTimeout(this, delegate () {
    //do something
}, 15000); //do not wait more than 15 seconds for lock

As you can see, usage is very similar but in second case we should never get deadlock, because code will simply throw exception if it can not get lock.

This code could be used in debug version when you stress test your multithreaded code, so you can see if you get any timeout exceptions (deadlocks occur). From stack trace you should be able to see which part of your code is problematic.

In production you can leave this or replace it back with standard lock statement.