I need to check for libdnet from configure.
I need to check for pugixml from configure.
I need to check for upsclient from configure.  'nut' package from MacPorts
on macOS does not have the header files. :-(  upsclient.h and parseconf.h

I'd like to have alerts when backups are failing for whatever reason.
This is a local test, and since I'm generally using 'dump' for backups,
I can just use the dumpdates file.  I kind of have to assume that
whatever is doing the backups is doing what it needs to do with respect
to full and incremental, so I can just look for the latest dump date for
a given filesystem.  The mcrover config file needs to accept a time
interval, and we'll post an alert if a dump has not completed within
that interval.  My backups are all daily, so in my case a reasonable
interval is something like 30 hours.

----------------------------------------------------------------------------
I need to be able to specify v6 versus v4 for service targets of
hosts.  It's reaosnable that a dual-stack host may run a service
on an IPv4 or IPv6 port but not both.

Now that I can send the config to an ostream, I'd like to create an
application that can generate a skeletal configuration based on the
local host's resources: ZFS pools, filesystems, disks and routes.
I should also be able to configure a skeletal server entry for localhost.

I'd also really like a way to discover pack member candidates: multicast?

I'd like a test for piwigo.  The example below gets me the list of
categories in JSON format from Randy's site, and I can check the size
of the "result":"categories" array in the JSON.

https://www.spcarsplus.com/piwigo/ws.php?format=json&method=pwg.categories.getList

I'd like a similar test for WordPress.  For example, this gets me the
title and author of the first 20 posts on my blog:

'http://www.rfdm.com/blog/?rest_route=/wp/v2/posts&_fields=title,author&per_page=20'

In terms of alerts...

It's kind of a shame to have many alerts for things like a web server
where all URLs will show up if the server is unreachable.  Random thoughts
here....

The per-URL alerts are of course necessary in many cases.  It's not uncommon
for a web app to talk to a database process, and if the database process
causes the web app to fail, we want an alert.  In the case of tests for
web applications, we can set a plain TCP port alert if connect() fails
and forget the remaining tests, but the downside is that we then lose the
'impact' of the connect() failure (the web server might have dozens of
important web applications at different URLs, or have part of a REST
interface fialing while other parts work.

This is an age-old NMS question.  You might need the detail, so it needs
to be carried around (consuming computing resources).  But during panic
triage, it's important to be able to zero in quickly on issues which carry
a high probability of being the root cause of multiple issues.  It's a
presentation feature.

If we start at the TCP transport layer and move upward, we have:

- can connect
  - [N] layers of application protocols.

For a typical web application, what's potentially visible from a remote
client:

  - can or can not connect
  - can or can not authenticate (https)
  - can or can not GET content
  - can or can not POST content (* I don't really want to implement this
    despite it not being diffult for several things)
  - received GET content indicates 'success' or 'fail', application
    dependent.

My plans for piwigo and WordPress are essentially the same:

  - connect
  - authenticate server certifticate (https)
  - GET content
    - if incorrect status, ignore content
    - if correct status, process content in app-specific manner to
      determine if GET response is OK (no alert) or FAIL (alert)

So we technically have 4 layers of failure:
  - failed to connect
  - failed to authenticate server certificate (https)
  - failed to GET content (http status)
  - got content but content is incorrect (application specific)

Ideally alerts of this type would be 'collapsible' when displaying.
If we can't connect to the same TCP tuple for any number of web
applications, I'd like to be able to see only a single TCP connect
alert.  The same is true for failure to authenticate.

Where things get interesting is collapsing across multiple alert
types, multiple hosts, etc.  We're moving into an era where machine
learning will be of use here.  I think the difficulty will be keeping
training data and inference current, but I don't think it's
insurmountable.  It's definitely surmountable for many of the things
any service provider needs.

However, there are still many useful non-DL methods to collapse alerts.
And the current intended scale of mcrover is dozens of monitored servers
per pack, not thousands.  I don't think I need inference here; some
fairly simple methods (especially in shallow networks) can go a long
way.  For example, if my next hop for a whole lot of services is
down, I can collapse all alerts for services that will be unreachable
when the next hop is down.  What sucks about this today is that 'down'
isn't as simple to determine as it was 20+ years ago: ICMP is often
rate-limited and sometimes disabled.  If it's ethernet (and it almost
always is), no ARP or IPV6 neighbor entry is a telltale, but the
timeouts can be quite lengthy.

ICMP will work in many cases.  In others we'll likely need a TCP connect()
test, assuming we don't want to send raw TCP packets.

Back to web apps...

I'd like a state machine with these states:
  Initial
  Connecting
  Authenticating (https cert)
  GettingContent (HTTP GET)
  CheckingContent (application specific)
  Done
  
A base class can have defaults for the first four states.  We only need
to override the last state.

----------------------------------------------------------------------------
I need alerts for Plex.  The REST API uses XML, which is kind of a bummer,
but an example:

dwm@www:/home/dwm% curl -k 'https://plex:32400/library/sections/?X-Plex-Token=bzAqYi6JHHd3KDP-bhNs'
<?xml version="1.0" encoding="UTF-8"?>
<MediaContainer size="1" allowSync="0" title1="Plex Library">
<Directory allowSync="1" art="/:/resources/movie-fanart.jpg" composite="/library/sections/3/composite/1651801829" filters="1" refreshing="0" thumb="/:/resources/movie.png" key="3" type="movie" title="Movies" agent="tv.plex.agents.movie" scanner="Plex Movie" language="en-US" uuid="ae96c3c9-4b97-4f57-bbff-505338d1f460" updatedAt="1651801783" createdAt="1480919308" scannedAt="1651801829" content="1" directory="1" contentChangedAt="142086" hidden="0">
<Location id="3" path="/zfs1/plex/Movies" />
</Directory>
</MediaContainer>

----------------------------------------------------------------------------

DONE - I now have web app alerting for WordPress and piwigo.

DONE - Use libDwmWebUtils, which replaces Poco with boost::beast.

DONE - Use libDwmCredence instead of libDwmAuth, which replaces Crypto++
       with libsodium.
       
DONE - Change HTTP tests to use poco, so I can unify HTTPS test too.

DONE - The config for HTTP tests should allow me to specify the expected
HTTP response code, as a single number, a set of numbers, a range
of numbers or any combination of these.
