Weird display (framebuffer?) issue.

I've got a 480x800 display in HDMI-1, and a 1920x1080 in HDMI-2. I normally keep HDMI-1 in an orientation where I have to set its rotation to "left" and it then becomes 800x480. But whether or not I rotate it, HDMI-2 shows the contents of HDMI-1 on the right side of the screen, but with the top line of pixels missing. I've tried numerous adjustments in arandr and in /boot/config.txt to no avail. Also of note is that anything I click on on the right screen, the actual click happens about 2 inches to the left of wherever the arrow is. If I move the arrow onto HDMI-1, it doesn't show up in the area of HDMI-2 where HDMI-1's screen is showing, but the effects of me clicking do. But where it gets really weird is that if I set the rotation of HDMI-2 to inverted, it works perfectly fine, other than it's annoying having my monitor upside down.

Pi 4 8GB model

Shows up as "Debian 12" on the X login screen, with a kernel of Linux pocket 6.1.55-v7l+ #1686 SMP Thu Oct 5 15:43:37 BST 2023 armv7l GNU/Linux

I'm using dtoverlay=vc4-kms-v3d as otherwise the Pi doesn't detect HDMI-2 at all, and HDMI-1 shows up with the name "default" instead

It feels like I need t o get the 1920 in the output below to change to 2720, but that is an illegal argument.

pi@pocket:~ $ fbset

mode "480x800"
    geometry 480 800 1920 1080 16
    timings 0 0 0 0 0 0 0
    accel true
    rgba 5/11,6/5,5/0,0/0
endmode

pi@pocket:~ $

I've confirmed that HDMI-2 "works fine" using any rotation but normal. :/

Oh, and the issue doesn't appear to happen in the login screen, but does happen as soon as I'm in xfce4 or spectrwm.

Moved HDMI-2 to the upper left of arandr and HDMI-1 to the right

and then HDMI-1 had the issue instead

If I move HDMI-1 below HDMI-2, it's all scrambled like it's way out of the buffer area

and of course, if I rotate HDMI-1 to any direction but normal, it's fine.