Well, I have a residential elevator, and it's important to be able to call for help if you ever get stuck in it, it's also a part of the fire codes. For the curious, the requirements are codified in ASME A17.1-2019/CSA B44-19 section 2.27 Car Emergency Signaling Devices. That's why my elevator has a good old hardwired phone headset inside its cabin.

Unfortunately, I don't have a hardwired phone line anymore, and I don't want to get one either. It's a pretty steam-punky, that just a pair of wires can connect you to the central phone office, and provide you with a trickle of power even in the event of a complete outage. But in reality, hardwired phone lines are pretty susceptible to damage. They are also an annoyance to maintain, and they are pretty expensive.

So for a long while, I've just been using a trusty Yeastar S20 PBX with an S2 FXS module. This module provides the “station” part of the phone interface, allowing the handset inside the elevator to place calls. The call was then routed through a VoIP provider (1-Voip in my case).

To safeguard against electricity outages, my Internet router is powered through a large UPS, good enough for 3–4 hours, it also feeds power into the network via a PoE-capable switch. The Yeastar box in turn is powered through it.

This works fine... But I have never trusted this system completely, because of a huge number of “moving parts”. In an event of a power outage, my ISP can go offline, or maybe my UPS can malfunction, or my VoIP provider can randomly block my account.

Finding a better solution: LTE

My first idea: 1. Add an LTE module to the PBX. 2. Sign up for a voice-only line. 3. Add a backup battery to power the PBX box in case of an outage.

Well, first I bought an LTE module on eBay for $400 (ouch!): Next, I tried to set up a cheapest-possible voice-only line with T-Mobile or AT&T. It turned out, that it would cost me more than $65 a month because “I'm on the unlimited plan”. Really, guys?

So I ordered a SIM card from Tello, an MVNO. They have really nice plans for exactly this use-case, for around $10 a month I can get a phone line with per-minute billing. I also got a backup 12V battery So I installed everything, and quickly set up the call routing via the LTE gateway.

It didn't work. The PBX couldn't place a phone call via that LTE module. I tried various permutations of options, and they all failed.

The module worked fine in data mode, though. I was able to set it up as a backup connectivity option for the PBX box. Yealink has really nice support for that, see the Failover Mode in their docs.

I could live with that, but this solution had still a bit too many “moving parts” for my liking. I still depended on the SIP provider for outbound access, I still needed the battery to hold, and I needed the LTE failover to work properly.

An even better solution: FXO+FXS

My attempts at setting up the connectivity highlighted that I'm really spending too much money on cellular lines that I barely use.

I have several tablets and phones, and they all have AT&T phone lines, and I even have two entirely unused lines for devices that I no longer have. I tried cancelling them, and AT&T kept me waiting for 40 minutes, offering various inane “deals” instead. This was the last straw, and I decided to move my secondary phones to an MVNO.

So I started browsing for options, and I stumbled on this: It's a phone gateway offered by US Mobile as a home phone replacement. It has a backup battery and it only costs $10 a month. So I ordered it, and also got one of the FXO+FXS modules from eBay: These modules have both a “station” port that allows connecting a phone headset, and an “office” port that allows it to connect to the PSTN (or a phone gateway).

I connected everything, and while in the process of testing the backup battery, I let it run out completely, resulting in the PBX box going offline. But to my surprise, when I tried to pick up the elevator phone, I got a dial tone! It turns out, that the FXO+FXS module ties both of its inputs together if it's offline. So a phone connected to its FXS input gets directly connected to the gateway on the FXO input.

This is perfect for me!

This allowed me to immediately simplify my setup, I no longer need a backup battery for the PBX box. So here's the final result:

TODO:

Here are some items I'd love to do eventually: 1. Add a periodic test call to verify that the gateway and SIP providers work correctly, and raise an alarm if the call fails.

#homesetup #sip #phone #todo

Discuss...

Alternative firmware

Buy firmware and get a license

Install the license

Set up the SIP settings

#homesetup #sip #videobell

Discuss...

So I've recently set up my alarm system to work with my home automation and with my security company. It works really well, and I'm delighted by the results.

Here's one problem, though. I like in-garage deliveries from Amazon, and I'd like to automatically disarm the garage partition when the delivery driver operates the garage door opener. And then re-arm the partition once the delivery driver leaves (of course, if the main partition is still armed).

Planning

First, I looked at doing everything using software only. I have an Envisalink card connected to my DSC alarm, so writing a HomeAssistant automation is fairly straightforward. I just

Ratgdo wiring

Assembling the synchronizer

Placing everything on a panel

Terminal blocks

Optocouplers

Power

Arduino programming

Testing

#dsc #homesetup

Discuss...

It's time to set up my new alarm system, after moving everything to a location downstairs and wiring up everything using Ethernet (see structured wiring for DSC). I have a DSC PC-1864 alarm, and here are some notes about setting it up. And it turned out to be not at all scary, once I found the right software!

First, I needed to ensure that the alarm is back in the factory configuration, this is easily done by shorting the PGM1 output with Zone 1 input (while the alarm unpowered) and then connecting the board to power for 10 seconds. After the reset, you can check that everything is working by getting into the installer mode using the default password (5555): "*8 5555". Now it's ready for the zone setup.

There is a nice blog from Chris Schuld detailing the alarm setup for his home. I did something similar before, but it's completely unintuitive and error-prone. But it turns out that DSC produces (a somewhat crappy) Windows-based software that can be used to set up their alarms using a visual UI.

It's called DLS-5, and it's a free download that is normally locked on the installer-only section of their website. Fortunately, it can be bought for $40 from https://www.credexalarmsystems.eu/en/dsc-software-dls5.html The panels are region-specific, so I made sure to get the North America (NA) version.

I also bought a PC-Link cable, CredexAlarmSystems sells them, but I found one much cheaper on eBay. I also bought one of the “PC-Link Adapter” cables from eBay for around $25, after failing to make the DLS work:

Ultimately, both cables worked fine, after I found (by accident) the correct way to connect them, see below for the details.

These cables are simple pass-through serial cables, so you should be able to just connect any other USB-serial adapter, but I had no luck connecting my RaspberryPi adapter with female ferrule-style pins.

Connecting the DLS

System setup

Setting up the zones

Setting up partitions

Envisalink

#dsc #homesetup

Discuss...

DSC alarm panels are usually a veritable rats' nest of wires and strange connectors, so I decided to try and clean up my panel by using structured wiring. This makes a lot of sense, standard Ethernet wiring is good enough for 2 amps for a single pair, which provides more enough power for the DSC panel and alarms.

My house uses a Legrand On-Q enclosure, so I used an AC1015 Network Interface Module to terminate the wires from sensors in individual zones. At this point, I spent some time deciding on the way to map the wiring from the DSC standard to the T-568A Ethernet pinout.

Security alarms typically use cables with 4 wires: Black, Red, Green and Yellow. Sometimes cables might have Blue wires instead of Green ones.

This is the Ethernet wiring mapping that I came up with:

T-568A	4-wire	Meaning
WhiteGreen
Green
WhiteOrange	Black	12V(–)
Blue	Red	12V(+)
WhiteBlue	Green (Blue)	KEYBUS
Orange	Yellow	KEYBUS
WhiteBrown
Brown

DSC uses the Black and Red wires for 12V DC power supply, so it's important to make sure that they are not connected to the same Ethernet pair (for example, do NOT connect them to blue and white-blue). This way, if you accidentally plug in a regular Ethernet device into a DSC plug, there won't be any magic smoke emitted.

Additionally, I used RJ11 phone jacks with 4-wire cable instead of the regular CAT6 cable for the zone-to-alarm wiring. It saves a bit of space and makes sure they are visibly distinct from Ethernet jacks. RJ11 jacks are perfectly compatible with the regular CAT6 Ethernet sockets.

Wiring the zones to a patch panel

The first step in organizing the mess was to terminate the wires from zone sensors in the network interface module. This turned out to be pretty easy, I just lined up the wires according to the mapping above and punched them in using a Keystone Punch Down Tool. It can be done with a thin screwdriver, but a proper punch down tool saves a lot of effort.

My alarm wiring shorts the Black wires with Green, and Red wires with Yellow at the door/window sensors. Then the sensor in turn shorts these two 2-wire bundles all together when it's closed. So all four wires end up shorted.

It means that if I connect all these zones to the alarm panel according to the wiring above, I'll end up with shorted +12V and -12V power lines. I have basically two ways to deal with it: I can leave the Black and Red wires (coming up from the sensors) disconnected inside the Network Interface Module, or I can use patch cables that don't have Black and Red wires to connect the door/window sensors to the alarm terminals.

Leaving the wires disconnected inside the Network Interface Module will make any future upgrade from passive sensors to powered motion sensors harder, but it's completely safe. Fortunately, it turned out that shorting the +12V and -12V is not a big deal for the DSC alarm panels, they simply shut down as a result and come back up once the fault is removed. So I decided to just use appropriate patch cables.

Here's how the alarm zone panel looks:

Using extender for zones

Another thing is that I wanted to move my alarm panel into the basement, out of my living room. This is mainly to get rid of an ugly metal cabinet in my food pantry, and also because my wiring cabinet was getting congested.

I added several conduits leading from the basement up into my living room, so there was plenty of space to just run the wires to the panel downstairs. But then I had an idea, I could use a zone expander and just run one wire from it to the control panel!

It worked well, I was able to use just one Ethernet cable to connect the zone expander with the main panel downstairs. Here's how the living room panel looks, and it also holds an Envisalink 4 board that exposes the DSC alarm state for my HomeAssistant: \ And this is the main alarm panel:

Super-tidy!

For comparison, this was the initial look.

Expander wiring

Zone expanders for the DSC alarm systems use the same KEYBUS standard for 4-wire cables. But we need two more modifications: 1. I also need to send the bell (siren) signal up from the main alarm panel. 2. I'd like to have some headroom for power. I have 3 keypads and 1 security camera powered by 12V from the expander, and this is just a bit too close to the maximum rated amperage for the CAT5 cable pairs. So I used up the remaining twisted pair for an additional power line.

This is the updated mapping for the expander-main connection:

T-568A	Expander	Meaning
WhiteGreen	Additional Red	MORE POWER
Green	Bell (–)	Siren
WhiteOrange	Black	12V(–)
Blue	Red	12V(+)
WhiteBlue	Green (Blue)	KEYBUS
Orange	Yellow	KEYBUS
WhiteBrown	Additional Black	MORE POWER
Brown	Bell (+)	Siren

As a note for myself, here's the zone mapping between locations and zones, they go in the descending order, from upstairs to downstairs:

Location	Expander Zone	Global Zone
4th floor door	8	16
3rd floor: balcony door and living room PIR	7	15
2nd floor balcony door	6	14
2nd floor bedroom PIR	5	13
2nd floor entry hallway PIR	4	12
2nd floor entry door	3	11
1st floor bedroom door and window	2	10
1st floor elevator door	1	9
garage PIR	Main Panel	8

PIR stands for Passive InfraRed, in other words, motion detectors.

#dsc #homesetup

Discuss...

Here's how I set up my blogging environment

Discuss...

Ratgdo (Rage Against The Garage Door Openers), is an OpenSource project that allows one to interface with the obfuscated protocol that the “smart” Chamberlain garage door openers are using instead of good old dry contacts interface. Ratgdo can also provide the said dry contact interface for your projects, or to provide remote capability to good old “dumb” garage door openers.

Making it look nice

Printing a box

Connecting connectors

Adding the dry contact interface

Installing and powering

How NOT to power it up

Backup battery with a controller

Using my alarm panel

Closing thoughts

Discuss...

Benchmarking Go and C# async performance

One large problem with async/await-based coroutines is that they are not very fast when compared with properly implemented fibers (aka “lightweight threads” or “goroutines”).

This is easy to demonstrate with a simple program:

package main

import (
	"log"
	"time"
)

func Calculate(val int) int {
	if val == 0 {
		return 0
	}
	res := Calculate(val - 1)
	res += 1
	return res
}

func main() {
	start := time.Now()
	var num int
	for i := 0; i < 1000000; i++ {
		num += Calculate(20)
	}
	diff := time.Now().Sub(start)
	log.Printf("Val: %d, %d(ms)", num, diff.Milliseconds())
}

Its output:

cyberax@CybArm:~/simp/bench$ go run test.go
2023/03/16 17:14:47 Val: 20000000, 32(ms)

I'm going to use C# to implement the coroutine-based version:

using System;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;

namespace ChannelsTest
{
    class Program
    {
        static async Task<int> Calculate(int val) {
                if (val == 0) {
                        return 0;
                }
                int res = await Calculate(val - 1);
                res += 1;
                return res;
        }


        static void Main(string[] args)
        {
                int num = 0;
                var sw = new Stopwatch();
                sw.Start();
                for (var i = 0; i < 1000000; i++) {
                        num += Calculate(20).Result;
                }
                sw.Stop();
                Console.WriteLine($"Result is: {num}, {sw.Elapsed.TotalMilliseconds:0.000}ms");
        }
    }
}

Its output:

cyberax@CybArm:~/simp/bench$ dotnet run -c Release
Result is: 20000000, 492,313ms

This is caused by the C# version having to allocate a heap object to store the stack frame for each level in the Calculate function. In contrast, Go simply uses a normal stack, that can be grown as needed.

Discuss...

Some years ago, AWS introduced the SSM (Simple Systems Manager) Agent. It's an agent that can be started on EC2 instances and perform multiple utility functions. Over the years, the SSM agent was added to all the major cloud-enabled Linux distribution AMIs, including Ubuntu, Amazon Linux, and RHEL. And AWS even added an option to automatically add SSM connectivity to all EC2 instances via Default Host Management!

SSM Agent supports a wide range of functionality. It can inventory running processes, apply patches, run shell commands, establish terminal sessions to EC2 instances, and even set up port forwarding.

The most significant advantage of the SSM Agent is its complete independence from the VPC settings. It uses an EC2 service, ssmmessages, and as a result, it can work just fine even in a VPC that doesn't have connectivity with the public Internet.

I was primarily interested in using this to set up port forwarding and, avoid using bastion hosts for SSH or PostgreSQL access.

Unfortunately, AWS's existing tooling around the SSM protocol is very clunky and can't be easily used in a composable standalone library. So I spent some time doing just that. The result of this work is Gimlet.

Looking at how SSM is supposed to be used normally

The normal way to use the SSM port forwarding is by using the AWS CLI with an optional Session Manager Plugin.

For example, to set up port forwarding to the instance i-0e3a964d49f28a5b8 and port 22 we need to run:

aws ssm start-session --target i-0e3a964d49f28a5b8 --document-name AWS-StartPortForwardingSession --parameters '{"portNumber":["22"], "localPortNumber":["56789"]}'

Starting session with SessionId: admin-0b6458385d2cffd35
Port 56789 opened for sessionId admin-0b6458385d2cffd35.
Waiting for connections...

The AWS CLI itself doesn't actually do anything but initiate the session, and the work of port forwarding is handled by spawning a background daemon:

cyberax@CybArm:~$ ps aux | grep session-manager-plugin
cyberax          43380   0,0  0,0 408636096   1488 s003  S+   10:01     0:00.00 grep session-manager-plugin
cyberax          43163   0,0  0,0 35315540  14816 s001  S+   10:00     0:00.27 session-manager-plugin {"SessionId": "admin-0b6458385d2cffd35", "TokenValue": "AAEAA......417bvh4OL", "StreamUrl": "wss://ssmmessages.us-east-1.amazonaws.com/v1/data-channel/admin-0b6458385d2cffd35?role=publish_subscribe&cell-number=AAEAAf2PldzWvh3EDdw8q7A0JB+nBIqkCvU+htAEPX0+D2QYAAAAAGPPc/9A2Ugc4EBbN5Qt9rCZMB9iBuYX6zUdShZndrZ5tvWh3g==", "ResponseMetadata": {"RequestId": "208ec786-7805-4965-91d7-8e6f7e95a603", "HTTPStatusCode": 200, "HTTPHeaders": {"server": "Server", "date": "Tue, 24 Jan 2023 06:00:31 GMT", "content-type": "application/x-amz-json-1.1", "content-length": "947", "connection": "keep-alive", "x-amzn-requestid": "208ec786-7805-4965-91d7-8e6f7e95a603"}, "RetryAttempts": 0}} us-east-1 StartSession pers {"Target": "i-0e3a964d49f28a5b8", "DocumentName": "AWS-StartPortForwardingSession", "Parameters": {"portNumber": ["22"], "localPortNumber": ["56789"]}} https://ssm.us-east-1.amazonaws.com

Yup. The parameters, including connection tokens and request metadata, are passed through the command line. Sigh.

The daemon is also quite a bit... hacky. It's just not written well, and can't really be used as a composable library inside your own application.

“Reverse engineering” the SSM port forwarding protocol

It's clear that the default implementation of session-manager-plugin leaves a lot to be desired. So we should just re-implement it! AWS is known for its pretty good documentation, so it should be simple, right?

The SSM port forwarding API calls are very eloquently documented by Amazon as special operations used by AWS Systems Manager. Which is about the total extent of the available documentation.

Fortunately, we do have the source code for both the server side and the client side. So we just need to read it and untangle its twisted web.

I documented the results of my investigation in Gimlet's README file.

In the next post, I'm going to demonstrate how Gimlet can be used to build a simple SSH proxy to allow passwordless access to EC2 instances.

Discuss...

What is Multitenancy

Even in the microservice world there's a common requirement to host data for many tenants. But first let's discuss what exactly is a tenant. When discussing multitenancy, various websites and some books give examples like this: a company that resells database access and needs to store data from multiple clients in one database. This is an extremely naïve example and it doesn't really reflect the actual reality.

For the purpose of this document, a tenant is a group of users that work within the same organization (or are somehow associated). For example, if we're making an enterprise chat application (a Slack clone, why not?) then a tenant would be an organization that subscribes for it. And the main goal would be to make sure that one tenant can't access the data from other tenants.

We likely still need to add some kind of access control within the tenant, but we absolutely need to make sure that data doesn't leak across the tenant boundary. Moreover, we should make it a goal to ensure that any bug in our code does not result in leaking data outside of the tenant. Basically, imagine how you can deal with the worst possible scenarios: arbitrary unlimited SQL injection, or a fully exploitable buffer overflow in server code.

PostgreSQL Row-Level Security

So with this in mind let's start designing the data access code. We're using PostgreSQL as our main data storage, so we'll be looking at securing it. We can't do database-per-tenant or schema-per-tenant partitioning as this will blow up the complexity of all the routine operations like database upgrade. Instead we'll be looking at row-level security.

Let's start with the schema and some sample data. I purposefully use human-readable names for tenants, in actual production code it's better to use something like uuid default uuid_generate_v4() instead (and probably for the orders table as well).

-- The application role
create role slack_app nosuperuser nocreatedb nocreaterole login password '123';

-- The tenants table
create table tenant (tenant_id varchar primary key, name varchar);
grant select on tenant to slack_app;

-- And a simple data table
create table orders (order_id int8 primary key, order_text varchar, 
    tenant_id varchar not null references tenant(tenant_id) on delete restrict);
grant select, insert, update, delete on orders to slack_app;

insert into tenant(tenant_id, name) values('HHL','Horns&Hooves Ltd.');
insert into tenant(tenant_id, name) values('CIA', 'Scary Government Agency');
insert into orders(order_id, order_text, tenant_id) values (1, 'Chairs', 'HHL');
insert into orders(order_id, order_text, tenant_id) values (2, 'Diamonds', 'HHL');
insert into orders(order_id, order_text, tenant_id) values (3, 'Killer Drones', 'CIA');

alter table tenant enable row level security;
alter table orders enable row level security;

So far so good. We can log into the database as slack_app and do the CRUD operations on tenants and orders.

Now we need to add some kind of security. This how-to guide has a nice tutorial, so we'll follow it.

create policy tenant_isolation_policy on tenant using (tenant_id = current_setting('app.current_tenant'));
create policy tenant_isolation_policy on orders using (tenant_id = current_setting('app.current_tenant'));

Now let's test it by logging in as slack_app and trying to do something:

cyberax@CybMac:/tmp$ psql --user slack_app slackapp
slackapp=> select * from orders;
ERROR:  unrecognized configuration parameter "app.current_tenant"

Good, we can't see all the data. Now let's try to change the tenant:

slackapp=> set app.current_tenant = 'HHL';
SET
slackapp=> select * from orders;
 order_id | order_text | tenant_id
----------+------------+-----------
        1 | Chairs     | HHL
        2 | Diamonds   | HHL
(2 rows)

Great! We can only see our own data. But there's one small problem, as nothing whatsoever stops an attacker that gained ability to do arbitrary SQL injection from doing this:

slackapp=> set app.current_tenant = 'CIA';
SET
slackapp=> select * from orders;
 order_id |  order_text   | tenant_id
----------+---------------+-----------
        3 | Killer Drones | CIA
(1 row)

There is no way to limit the SET operations in PostgreSQL to be one-time only. Also there are no ways to prohibit running SET commands altogether, or at least limit them to a set of whitelisted options.

Tokenizing Everything

One possible solution is to use unguessable tenant names (e.g. UUIDs), so this way the attacker likely won't know the other tenants' IDs right off the bat. But this is not a good solution, any tenant ID leak would give an attacker access to all the tenant's documents.

But this approach seems to be on the right track. What if instead of tenants we use one-time tokens? These tokens can be populated by a small highly-secure service and passed to the main application.

Let's try it! We need to create a table for the tokens and a stored procedure to check them:

create table token(token varchar primary key, tenant_id varchar not null references tenant(tenant_id) on delete restrict, valid_until timestamp);

create or replace function get_tenant()
returns varchar language plpgsql security definer
as $$ declare 
  tenant_res varchar;
begin
select tenant_id into tenant_res from token where token = current_setting('app.token', true) and now() < valid_until;
return tenant_res;
end $$;

Some explanations: security definer modifier means that the function is always invoked with the permissions of the user that defined it (the superuser in this case). This is necessary because we absolutely DO NOT want to give the slack_app user permissions to do select on our tokens table.

The rest is straightforward, we need to modify the row-level security policy on the tables and insert some test tokens.

drop policy tenant_isolation_policy on tenant;
drop policy tenant_isolation_policy on orders;

create policy tenant_isolation_policy on tenant using (tenant_id = get_tenant());
create policy tenant_isolation_policy on orders using (tenant_id = get_tenant());

-- Insert some test tokens (they MUST be unguessable cryptographically random strings in a real application)
insert into token (tenant_id, token, valid_until) values ('CIA', 'token-high', now() + interval '2 hours');
insert into token (tenant_id, token, valid_until) values ('HHL', 'token-low', now() + interval '2 hours');

And now let's test it!

cyberax@CybMac:/tmp$ psql --user slack_app slackapp
psql (13.3)
Type "help" for help.

slackapp=> select * from orders ;
 order_id | order_text | tenant_id
----------+------------+-----------
(0 rows)

slackapp=> set app.token = 'token-high';
slackapp=> select * from orders;
 order_id |  order_text   | tenant_id
----------+---------------+-----------
        3 | Killer Drones | CIA
(1 row)

slackapp=> set app.token = 'token-low';
slackapp=> select * from orders;
 order_id | order_text | tenant_id
----------+------------+-----------
        1 | Chairs     | HHL
        2 | Diamonds   | HHL
(2 rows)

And this is exactly what we want! The limited time tokens provide authorization to access the data for individual tenants. The tokens can be generated by a small service and communicated to the app over a secure channel. And there's nothing an attacker can do without knowing a token.

Performance

Row-level security is implemented as a hidden where clause, that is visible in the explain statement:

slackapp=> explain select * from orders;
                       QUERY PLAN
---------------------------------------------------------
 Seq Scan on orders  (cost=0.00..222.62 rows=4 width=72)
   Filter: ((tenant_id)::text = (get_tenant())::text)
(2 rows)

This hidden clause needs to be taken into account when designing queries and indexes.

Discuss...