Recently xssfox posted that xrandr is able to do incredible things, for example render the image diagonally on the screen.

What if you could do the same, but automatically, in real time?

What have I just seen

You saw a computer monitor on a pivot that levels the displayed output when the monitor is turned. It's not much different than what smartphones do when set to alternate between the portrait and the landscape mode when the phone is rotated. The difference is that smartphones are hand-held devices and allow only 90° rotations. Otherwise, it would take enormous effort to keep your screen still when using the phone.

However, computer screens usually rest on desks and don't move much. Why not allow them turn with more fine precision?

How did I get this idea

I've been involved in open-minded discussion with my friends around following meme.

Source: Reddit

We were exchanging ideas about new ways of working with code. For example, what if there was a programming language in which parallel operations would be also parallel in the syntax? You'd need a wide monitor. What is the longest section of a rectangle? It's a diagonal, but it's unfortunately tilted, so you'd need to rotate the monitor. Then I thought that I have LSM303AGR accelerometer on micro:bit v2 board and I could use attach it to the monitor to track the orientation of the screen. Probably it's gigantically impractical idea, but what can go wrong?

Measuring monitor orientation

The accelerometer measures acceleration in 3 dimensions. Even when the sensor is still, you're getting acceleration readings, because sensor and all matter on Earth is experiencing gravitational acceleration towards the center of the Earth's mass. The sensor outputs are in mg (milli g's), so the calibrated sensor aligned on one axis will show value 1000, which corresponds 1 g.

The 3-dimensional math is more difficult than 2-dimensional one, but in case of monitor orientation I've been fortunate, because my monitor had pivot that rotated only around one axis. Because of that, I could ignore readings from one axis at all. In two dimensions rotation of the monitor can be then visualized in a Cartesian plane.

What you see are the plotted readings from the sensor when the board was rotated by 90 degrees. As you see, the readings are noisy, partially because of the inaccuracy of the sensor and maybe a unsteady hand.

Sending the data to the PC

I went for the simplest available method of data transfer from microcontroller device to a PC - serial communication over UART (or precisely its UARTE variant with additional capabilities). The board has UART-USB bridge that allows data transfer over an USB cable connected to the PC.

UART can transfer byte payloads and one of the problems to solve was the data layout. Serial data is just a stream of bytes with no indication how to interpret the data at the receiver end. Here we're sending 3 integers, but when converted to bytes, they can be interpreted also as ASCII text. If I wanted to use a delimiter to separate these 3 integer values, it might happen that one of the values contains a byte sequence of my delimiter. However, since this is a hack-for-fun project, I made some shortcuts when it comes to serialization.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

fn format_row(x: i32, y: i32, z: i32) -> heapless::Vec<u8, 256> {
    let mut str_buff = heapless::Vec::<u8, 256>::new();
    str_buff.extend(x.to_be_bytes());
    str_buff.push(',' as u8);
    str_buff.extend(y.to_be_bytes());
    str_buff.push(',' as u8);
    str_buff.extend(z.to_be_bytes());
    str_buff.push('\n' as u8);
    str_buff
}

In the code above I separated values with , character and terminated with a line break.

1

-1008,100,52\n

The data receiver (written in Python script with pyserial) was configured to split data by known delimiter and interpret values as big endian signed integer.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

def from_raw(bytearr: bytes) -> int:
    return int.from_bytes(bytearr, byteorder="big", signed=True)

# (...)

line = ser.readline()
# Ignore the last line break byte.
line = line[0:-1]
try:
    (x_raw, y_raw, z_raw) = line.split(b",")
    x = from_raw(x_raw)
    y = from_raw(y_raw)
    z = from_raw(z_raw)
except ValueError:
    # (...)

The proper solution would be to use fixed length for each value or utilize Protobuf for serialization and deserialization of the whole payload, but in the case of this project ignoring ambiguous data was good enough.

Calling xrandr

Once data was deserialized, it needed to be converted to a for understood by xrandr. First with the help of handy atan2 function I obtained the angle of inclination of the monitor in radians.

1
2

# Coordinate arguments swapped on purpose here
angle = math.atan2(x, y)

Next step was using the angle in arguments to xrandr, with formula used by xssfox.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

def make_transform_arg(angle_rad: float) -> str:
    def fmt(num: float) -> str:
        return "{:.4f}".format(num)

    a = math.cos(angle_rad)
    b = -math.sin(angle_rad)
    # "Heuristic" adjustment that shifts screen to the left.
    # Works fine enough in my resolution (2560x1440).
    c = angle_rad * 1000
    d = math.sin(angle_rad)
    e = math.cos(angle_rad)
    f = 0
    return f"{fmt(a)},{fmt(b)},{c},{fmt(d)},{fmt(e)},{f},0,0,1"

Results

As visible in the video, data provided by the sensor is accurate enough to give a nice experience of automatically leveling screen output whenever monitor is tilted. I'm still not sure if this is the future of human-computer interaction, but I have really high hopes!

Some practical problems solved

• Sometimes x, y, z integer values could be interpreted as "," characted from a byte payload. However accelerometer produced readings with 1 Hz frequency and it was just fine to skip these values when it was not clear how to interpret the data.
• Noise in the accelerometer readings caused the screen position to be updated too often, which made the screen flicker too much. Solved by ignoring angle changes not exceeding ±20% and also calculating average of some previous angles.
• Screen was shifted to the right after the transformation. I figured that it can be compensated by shifting left by the value proportional to the monitor angle.

Links

• Source of the project: Github spirit-level
• Original post from xssfox
• Great introduction to embedded programming in Rust