Tutorial
Install
See Installation page to access Ackal's container images and to register.
Console
Ackal includes a web-based Console that provides administrative functionality including login and creating a subscription to Ackal.
Ackal uses Firebase Authentication. This service supports various OAuth providers and enables you to choose your preferred provider(s) to login to Ackal. Once you are signed in, you can download your credentials as a JSON file to configure ackalctl
.
Ackal uses Stripe. This service is a trusted payments service and enables you to subscribe to Ackal. Even if signed in, you will not be able to use Ackal until you purchase a subscription.
Console displays user information and access to additional functionality:
Login
Ackal supports several federated identity providers
NOTE contact us if you want to associated different identities with your Customer ID.
Download
Clicking DOWNLOAD
will download a Credentials file (credentials.json
) to your machine.
ackalctl
looks for this file in ${HOME}/.config/ackalctl/credentials.json
.
Subscribe
Clicking SUBSCRIBE
will redirect you to Stripe and to its subscription signup page.
You may enter your billing details on this page to subscribe to Ackal.
Billing
Clicking BILLING
will redirect you to Stripe and to your subscription details page.
You may view details of your subscription and cancel your subscription on this page.
Credentials
Ackal's Console provides a way for you to Download your credentials as a JSON file (typically credentials.json
).
ackalctl
stores user credentials in ${HOME}/.config/ackalctl/credentials.json
.
You will need a Customer ID. A Customer ID represents one (individual) or many (organizational) users.
After subscribing to Ackal, you will need to request a Customer ID before you may begin creating health checks against your gRPC services.
NOTE Once created in Console, your credentials will expire within 60 minutes. Once you begin using them with ackalctl
, it will renew them for you indefinitely. If you need to manually refresh your Credentialsd in Console, log out and back in.
Domains
You are permitted to create health checks against servers on
- Public Domains (e.g. Google Cloud Run
- domains that you have verified that you control.
Before using domains that you own/control, you will need to generate a verification code and then add a DNS record to your domain to verify the ownership to Ackal:
DOMAIN="[YOUR DOMAIN]"
# Create the domain's record
ackalctl \
--domain_name=${DOMAIN} \
create domain
Either:
DOMAIN="[YOUR DOMAIN]"
# Get the domain's record
ackalctl \
--domain_name=${DOMAIN} \
get domain
NOTE The result includes a fieldverified
that will befalse
until the domain has been verified.
Or:
DOMAIN="[YOUR DOMAIN]"
# Get the domain's verification code
VERIFICATION=$(\
ackalctl \
--domain_name=${DOMAIN} \
--format=json \
get domain | jq -r .verification) \
&& echo ${VERIFICATION}
And then ensure that your DOMAIN
DNS records include:
Name | Type | Data |
TXT |
"ackal-verfication={VERIFICATION}" |
NOTE The requirement forNAME
varies by DNS provider. Some permit theNAME
field to be left blank, others require e.g.@
as a placeholder. Consult the provider's documentation.
You may use Linux tools (e.g. dig
or nslookup
) or Google's Admin Toolbox Dig
to verify that the TXT
record has been created correctly.
When you're confident the record has propagated, you can instruct Ackal to verify the domain:
DOMAIN="[YOUR DOMAIN]"
ackalctl \
--domain_name=${DOMAIN} \
verify domain
You can confirm that the domain has been verified successfully:
DOMAIN="[YOUR DOMAIN]"
# Get the domain's record
ackalctl \
--domain_name=${DOMAIN} \
get domain
NOTE The result includes a fieldverified
that will betrue
if Ackal was able to verify the domain.
Health Checks
Once you have ackalctl
installed, have logged in and downloaded your credentials, and have acquired a Customer ID, you're ready to begin creating health checks against your gRPC services.
You'll need gRPC services that implement the gRPC Healthcheck Protocol. If you do not have your own gRPC services to test, there is an example gRPC endpoint for you to use:
healthcheck-server.ack.al:443
You can list the service's implemented on this endpoint and you can invoke one of the Health Checking Protocol's methods using a general-purpose tool such as gRPCurl to confirm that it is working correctly:
ENDPOINT="healthcheck-server.ack.al:443"
grpcurl \
${ENDPOINT} \
list
grpc.health.v1.Health
grpc.reflection.v1alpha.ServerReflection
ENDPOINT="healthcheck-server.ack.al:443"
grpcurl \
-d '{"service":"grpc.health.v1.Health"}'
${ENDPOINT} \
grpc.health.v1.Health/Check
NOTE Per the Health Checking Protocol, you may optionally specify the gRPC service name to be checked. The endpointhealthcheck-server.ack.al
not only implements the Health Check Protocol asgrpc.health.v1.Health
but -- as shown above -- for convenience, it usesgrpc.health.v1.Health
as a service that you can health check. You may also use the empty string (""
) or omit the data (-d
) entirely to check the endpoint's overall health.
NOTE When you reference these gRPC service (using--endpoint
withackalctl
, it is important that you not prefix thehost:port
value with a scheme (i.e. nohttps://
) and you must include the port (i.e.:443
). Web browsers generally assume:443
when you're usinghttps
but this assumption is not valid with gRPC.
To test that ackalctl
is working correctly:
ackalctl list locations
Yields:
ID Name
las-vegas Las Vegas
london London
los-angeles Los Angeles
salt-lake-city Salt Lake City
the-dalles The Dalles
tokyo Tokyo
NOTEackalctl
supports JSON and YAML output formatting. Simply append--format=json
or--format=yaml
to any command, e.g.ackalctl --format=json list locations
:
{
"locations":[
{"id":"las-vegas","location":{"name":"Las Vegas"}},
{"id":"london","location":{"name":"London"}},
{"id":"los-angeles","location":{"name":"Los Angeles"}},
{"id":"salt-lake-city","location":{"name":"Salt Lake City"}},
{"id":"the-dalles","location":{"name":"The Dalles"}},
{"id":"tokyo","location":{"name":"Tokyo"}}
]
}
Then create a health check:
ENDPOINT="healthcheck-server.ack.al:443"
PERIOD="120s"
LOCATION_IDS="las-vegas,the-dalles"
SERVICES="ServiceA,ServiceB"
ackalctl \
--endpoint=${ENDPOINT} \
--period=${PERIOD} \
--location_ids=${LOCATION_IDS}
--services=${SERVICES}
create check
NOTE Health checks are uniquely identified by the service endpoint and the period.
Returns:
id: {ID}
Then:
ackalctl \
--endpoint=${ENDPOINT} \
--period=${PERIOD} \
get check
Returns:
endpoint: {ENDPOINT}
period: {PERIOD}
locationIDs: {LOCATION_IDS}
services: {SERVICES}
enabled: false
Or:
ackalctl list checks
Returns:
Endpoint Period Location(s) Service(s) Enabled
${ENDPOINT} ${PERIOD} ${LOCATION_IDS} ${SERVICES} ${ENABLED}
You may delete health checks:
ackalctl \
--endpoint=${ENDPOINT} \
--period=${PERIOD} \
delete check
NOTE Because Healthchecks are uniquely identified by the service endpoint and the period, both values are required to delete a health check.
Metrics
Ackal publishes Prometheus metrics for the health checks that you create. The simplest way to interact with your health checks' metrics is to use Prometheus (and possibly AlertManager and Grafana) to record, alert and graph them.
Ackal provides 2 proxies (ackal-http-proxy
and ackal-file-proxy
) that perform service discovery for Prometheus. Running either (or both) of these proxies locally, they will leverage your credentials to securely access your Healthcheck so that their metrics may be scraped by Prometheus.
You will need to have at least one health check:
ackalctl \
--endpoint=${ENDPOINT} \
--period=${PERIOD} \
get check
Returns:
endpoint: {ENDPOINT}
period: {PERIOD}
locationIDs: {LOCATION_IDS}
services: {SERVICES}
enabled: false
OPTIONALAckal's proxies discover your health check endpoints using an Ackal service called
listr
. You may uselistr
directly using a tool likecurl
. The following commands usecurl
,jq
andawk
to show how you can interact withlistr
directly:CUSTOMER="[YOUR CUSTOMER ID]" TOKEN=$(more ${HOME}/.config/ackalctl/credentials.json | jq -r '.jwt') LISTR="https://listr.ack.al:443" curl \ --silent \ --request GET \ --header "Ackal-Customer: ${CUSTOMER}" \ --header "Authorization: Bearer ${TOKEN}" \ ${LISTR}/?format=prometheus \ | jq -r .
If you omit the query string (
?format=prometheus
), the resulting JSON will be in Ackal's native format.This command lists your Ackal health check service endpoints in a Prometheus-friendly format:
[ { "targets": [ "{url}" ], "labels": { "endpoint": "{endpoint}", "location": "{location}", "period": "{period}" } } ]
You can then
curl
one of theendpoint
values returned, to query the specific health check for its metrics. This time we'll useawk
to filter the Prometheus metrics to those prefixedackal_exporter_code
:CUSTOMER="[YOUR CUSTOMER ID]" TOKEN=$(more ${HOME}/.config/ackalctl/credentials.json | jq -r 'jwt') ENDPOINT="[ONE OF THE ENDPOINT VALUES]" curl \ --silent \ --request GET \ --header "Ackal-Customer: ${CUSTOMER}" \ --header "Authorization: Bearer ${TOKEN}" \ ${ENDPOINT}/metrics \ | awk '\^ackal_exporter_check {print}
It is more convenient to configure a Prometheus server (or equivalent) to perform service discovery of your health checks. To do this, you will need to:
- Run
ackalctl-http-proxy
- Run Prometheus
- Optionally: run Grafana
Grafana
You can interact with Ackal's health check metrics through Grafana.
VERS="9.2.3"
docker run \
--interactive --rm --tty \
--net=host \
docker.io/grafana/grafana-oss:${VERS}
NOTE In order for Grafana to access Prometheus, Grafana must (also) be bound to your host's network with --net=host
.
You should now be able to browse Grafana (http://localhost:3000
).
The default username|password are both admin
.
You will want to add Prometheus as a datasource (http://localhost:3000/datasources
). The URL will be the default value (http://localhost:9090
) but you must type this in. The "Save & Test" to confirm that it works.
Then you can create a new Dashboard (http://localhost:3000/dashboard
) and for the value of Metrics Browser
, you can use:
avg
without(
endpoint,
job,
status,
)
(ackal_exporter_check_histogram_ms_bucket{
job="ackalctl-http-proxy",
})
You can import an example Ackal Check Dashboard that includes dropdowns that enable you to choose which Check and Location you would like to view.
If you'd prefer to build something for yourself based on the query shown above, you'll want to filter by a specific instance
and consider using Grafana's Histogram visualization tool.
Frequently Asked Questions (FAQ)
Public Domains
Ackal supports the creation of health checks against services running on public domains. These are domains -- often provided by cloud providers -- that are used to host services but which the developer does not own or control.
For this reason, health checks created against these domains are permitted without the user first needing to verify ownership of the domain.
For public (non-authenticated) services on these domains -- as with any other tool -- Ackal is able to access any service hosted on these domains.
Ackal supports the following public domains:
Domain | Service | Provider |
appspot.com |
App Engine | |
cloudfunctions.net |
Cloud Functions | |
a.app.run |
Cloud Run | |
fly.dev |
fly.io | Fly |
Ackal Proxies and TLS
When you configure Prometheus to use ackalctl-file-proxy
or ackalctl-http-proxy
you must use http://listr.ack.al:443
(http
but 443
) as the proxy_url
value URL for Ackal’s listr
service even though Ackal's listr
service uses TLS (https
).
This is to simplify your use of the proxies as it saves you having to configure between Prometheus and the proxy. Instead Prometheus uses plaintext http
to communicate with ackalctl-http-proxy
but the proxy itself communicates with Ackal using encrypted https
.
For this reason, you should not run ackalctl-http-proxy
outside your secure network perimeter.
What is gRPC Health Checking protocol?
Health checks are used by clients to determine the state of health of a server. The server's health check implementation should accurately report on the ability of the server to respond to client requests. For example, if the server depends on a database to respond to clients, if the database is down, the server is unhealthy and should report itself so. Often, particularly with HTTP/REST APIs, a server is deemed healthy if e.g. GET
'ting a method's endpoint returns a 200.
Because gRPC is a protocol layer above HTTP, it's not possible to use HTTP to check on a gRPC server's health. Additionally, because gRPC servers implement arbitrary sets of methods, there's no guarantee that every gRPC server will implement a health checking method, let alone that they will agree on the method name and request and response message types.
For these reasons, the gRPC Health Checking protocol is a de facto standard. If gRPC servers implement the Health Checking protocol, then clients know that the server will implement a grpc.health.v1
package with a Health
service and with Check
and Watch
methods both accepting HealthCheckRequest
's and returning HealthCheckResponse
's.
You can confirm whether a gRPC service implements the Health Checking protocol by listing its services or by invoking one of the Health Checking protocol methods directly:
ENDPOINT="healthcheck-server.ack.al:443"
grpcurl \
${ENDPOINT} \
list
grpc.health.v1.Health
grpc.reflection.v1alpha.ServerReflection
ENDPOINT="healthcheck-server.ack.al:443"
grpcurl \
${ENDPOINT} \
list grpc.health.v1.Health
grpc.health.v1.Health.Check
grpc.health.v1.Health.Watch
ENDPOINT="healthcheck-server.ack.al:443"
grpcurl \
-d '{"service":"grpc.health.v1.Health"}'
${ENDPOINT} \
grpc.health.v1.Health/Check
Ackal requires the gRPC Health Check protocol be implemented by gRPC services so it can provide health checks.