Wednesday, 21 October 2020

OLOy 64GB RAM Kit Review

Oloy RAM Kit.md

OLOy 64 GB WarHawk “Platinum Special Edition” 3200 Mhz Memory Kit

Product Image 4

Who TF is OLOy

A newcomer to the memory market, OLOy is apparently a trademark of “CHIPSET TECHNOLOGY HOLDING LIMITED.” Established in 2018, they appear to currently only offer a small range of DDR4 memory. Their basic and empty looking website claims they will offer SSD and IC products, but no mention of what they are or will be.

It seems they have started selling their memory modules on newegg and Amazon. I purchased my kit from Amazon UK for £199 GBP, a comparable kit from Corsair was £245 at the time of purchase so this was a substantialy lower price for the same speed kit. Since I wanted the cheapest kit I could find for a home lab “server” PC, this fit the bill pretty well.

Manufacturer Specifications

The kit is sold as a 64 GB (32 GB x 2) 3200 MHz kit at 16-20-20-38 with XMP and “limited lifetime warranty.” Since the brand is so new, I’m not holding out on that lifetime warranty. I purchased from Amazon UK so if there is a problem, as the retailer in the UK they are obligated to repair, replace or refund if the product is faulty (6 months from purchase.)

Out of the box

From the images on Amazon, this RAM looks pretty awful, I’d probably rather a simple rectangular design, but they were cheap so I don’t care too much about looks. In person they are actually not as bad as I thought. The look is pretty gamer but it’s not too bad.

The heatsink is painted metal with some plastic parts, the LED window / light spreader is a frosted white plastic.

Product Image 1
Product Image 2
Product Image 3

I switched from my existing DDR4-3000 Corsair Vengeance RGB Pro kit (at 3200 MHz) without adjusting settings and the system failed to boot. Switching back and then resetting RAM XMP to default and speed to Auto allowed the system to boot with the new modules. Once booted, I enabled XMP Profile 1 which automatically set the frequency to 3200 MHz with the manufacturer’s timings.

BIOS 1 BIOS 2

CPU-Z indicates the kit is running at 3200 MHz (DRAM Frequency 1600 MHz x 2) with 16-20-20-38 timings as advertised. SK Hynix DRAM is detected. Interestingly with 32 GB per module it is still a Dual rank module.

CPUZ Screenshot CPUZ Screenshot 2

AIDA64 shows the memory modules as “Chun Well Tech.” branded.

AIDA Modules

Performance

A couple of quick performance tests

Passmark Screenshot

AIDA Screenshot

Basic overclocking tests

A very quick attempt at overclocking failed miserably. Upping the DRAM frequency to DDR4-3400 in the BIOS resulted in a failed POST. I had to revert to my original Corsair RAM to get the system to boot so I could reset the RAM frequency. I didn’t change the timings or increase voltage, so loosening timings or increasing voltage may help. If you get a failed POST with this RAM you may need to reset your BIOS or switch to a known good memory kit to reset the XMP / RAM frequency.

Reliability

I performed a burn in using memtest86 overnight and… oh, oh dear!

The kit passed one full test but on the second and third tests a single error occurred. This might be a sign of a bad chip, or that the memory is getting too hot. The airflow in my case is pretty good and there are fans blowing air over both modules. It could be a sign that the memory is running really close to the wire. It’s not likely that these errors would occur in normal use, but still a worrying sign.

Memtest86 Results
Memtest86 Oloy Kit on Ryzen 3700x at 3200 MHz


Here’s another run at 3000 MHz to see if the memory is just rated too high. Unfortunately this also failed with the at the same address.

Memtest86 3000 Mhz Results
Memtest86 Oloy Kit on Ryzen 3700x at 3000 MHz


I tested on my “server” PC (which is just a desktop Inspiron) at 2400 MHz. Thankfully this passed 4 full passes.

Memtest86 2400 Mhz Results
Memtest86 Oloy Kit on i3-7100 at 2400 MHz


Since this test passed fine, I thought it could be an issue with the memory controller on my Ryzen PC. I ran this memory at 2666 MHz on my Ryzen desktop and it passed a couple of full tests just fine.

Memtest86 2666 Mhz Results
Memtest86 Oloy Kit on Ryzen 3700x at 2666 MHz


Here’s a test with my Corsair 32 GB DDR4 3000 MHz memory kit running at 3200 MHz to confirm the memory controller on my Ryzen can run at 3200 MHz. This kit has tighter timings than the Oloy kit, even overclocked to 3200 MHz.

CPU-Z Corsair Timings

Memtest86 Corsair 3200 Mhz Results
Memtest86 Corsair Kit on Ryzen 3700x at 3200 MHz


RGB lighting options

The Amazon listing states compatibility with various motherboard RGB controllers including Asus, MSI, Gigabyte and ASRock. Since my motherboard is MSI, I can only test with this software.

The software options with MSI Mysic Light are pretty limited, but it appears to support all the functions here. Some of them are demoed below.

This is the initial configuration - “rainbow mode”

RGB Demo Image

Here are a few of the options (Image links to YouTube)

RGB Demo

And here’s how the RAM looks in my system (Image links to YouTube)

In System Demo

Compatibility

I have tested the kit in my AMD B450 system with a Ryzen 3700x, CPU at stock speed, memory at the manufacturer supplied 3200 MHz XMP profile. Most of the above performance and reliability tests were in this system at these settings, so you can therefore expect this kit to operate the same in similar configurations. I might just have a bad kit causing the single memory error in memtest or it might be a sign that this RAM should not be rated for these speeds.

I have also tested in my home lab “server” PC - A Dell Inspiron 3668. This is running an i5 7400 at stock settings and therefore memory speed is limited to 2400 MHz. This system is only rated for up to 16 GB of memory, but since these high capacity modules weren’t available at the systems launch date, higher than 16 GB has not been validated by Dell. The system does POST and detect all 64 GB with this kit.

Conclusion

For a cheap memory kit from an unknown manufacturer this ticks some of the boxes for me. It almost operates as advertised, looks okay and passed my overnight burn in test (at a lower than advertised speed.) I wouldn’t expect anything in the way of overclocking this kit! If you’re going to run the RAM at it’s rated speed, I’d make sure to run a memtest pass to make sure there aren’t any horrific errors! If you’re worried and want reliable RAM at 3200 Mhz then it might be worth the extra 25% or so to get piece of mind. A comparable kit from Corsair was £245 at the time of purchase.

Since my use case was to run this RAM at 2400 MHz, I literally wanted the cheapest 64 GB kit that I could get. Even 2400 MHz kits from Crucial were £220+ at the time of writing. This kit performs fine at 2400 MHz and for the price it’s fine. I’ve been running this kit in my “server” running many virtual machines with no issues for the last week. Since my use case is in this machine, I won’t be sending it back for this issue.

If you want to buy this kit on Amazon, click here this is an affiliate link which will give me a small kick back. This review is not sponsored or affiliated with OLOy. I purchased the kit myself.

Written with StackEdit.

Saturday, 17 October 2020

Multicast packets dropped on OpenWRT VLANs

Multicast issue on OpenWRT.md

Multicast issue on OpenWRT

Issue

After setting up my hacked switches with VLANs, I wanted to create a pfSense cluster for more resilient internet access. pfSense uses CARP (closely related to VRRP) which uses multicast to detect when a router or interface goes down. When a router is in CARP master mode, it constantly sends out multicast packets to 224.0.0.18. If another router is in backup mode and cannot see the packets arriving from the master, it will attempt to become the master node.

The issue I was seeing was that the multicast packets weren’t arriving and therefore both nodes came up as master on all but one of the CARP IPs. The OpenWRT configuration I had initially only passed multicast on VLAN id 1, all multicast packets bound for VLAN id 1 were echoed to all ports on both switches. All of the multicast packets for the other VLANs were dropped.

Here is my original /etc/config/network configuration file:

Here’s how the CARP interfaces look. The only working network is LAN which is on VLAN id 1

Primary

Master CARP

Secondary

Backup CARP

It’s important to know that the VLANs all work and pass regular unicast and broadcast traffic just fine. Only the multicast CARP packets are not getting through.

Troubleshooting Steps

Packet capture on pfSense

Attempting to capture CARP on any of the affected VLANs shows no packets arriving.

CARP Packet Capture from pfSense

Nothing!

Nothing captured

tcpdump on the switch

Shows CARP packets arriving on the port destined for affected VLANs (Two IP ranges here are coming in on an untagged port on VLAN 9.) Also notice the CARP packets from VLAN id 1 are being echoed onto this port.

tcpdump -i eth1 -c 10 -n -e -T carp carp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
20:34:39.400955 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:34:39.401219 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 192.168.1.1 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 authlen=7 counter=13620309968023688989
20:34:39.401370 00:00:5e:00:01:0f > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 10.1.15.2 > 224.0.0.18: CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 authlen=7 counter=13306912573729581052
20:34:40.415665 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:34:40.415871 00:00:5e:00:01:0f > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 10.1.15.2 > 224.0.0.18: CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 authlen=7 counter=13306912573729581052
20:34:40.416211 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 192.168.1.1 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 authlen=7 counter=13620309968023688989
20:34:41.419605 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:34:41.419810 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 192.168.1.1 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 authlen=7 counter=13620309968023688989
20:34:41.419956 00:00:5e:00:01:0f > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 10.1.15.2 > 224.0.0.18: CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 authlen=7 counter=13306912573729581052
20:34:42.429564 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
10 packets captured
12 packets received by filter
0 packets dropped by kernel
4 packets dropped by interface

tcpdump on a linux endpoint

Shows CARP packets only for VLAN 1, this Linux machine is on an untagged port on VLAN 9.

# tcpdump -i eth0 -c 10 -n -e -T carp carp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:37:24.422703 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:37:25.438853 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:37:26.458870 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:37:27.522734 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:37:28.560403 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:37:29.625900 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:37:30.635363 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:37:31.645144 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:37:32.667945 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:37:33.717977 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
10 packets captured
10 packets received by filter
0 packets dropped by kernel

Resolution

I noticed on the switch that the bridge interface was configured only on VLAN id 1 since it’s member adapters were all fixed to vid='1'. My fix was to create a bridge interface and eth0-3 vlan interfaces in each VLAN that I wanted the multicast packets to work on.

The changed parts in /etc/config/network are the new bridge interfaces:

config interface 'lan2'
    option type 'bridge'
    option force_link '0'
    option igmp_snooping '1'
    option ipv6 '0'
    list ifname 'vlan_eth0_2'
    list ifname 'vlan_eth1_2'
    list ifname 'vlan_eth2_2'
    list ifname 'vlan_eth3_2'
    option ip6assign '0'
    list pppoerelay ''

And the new vlan_ethX_xx interfaces:

config device 'vlan_eth0_2'
    option type '8021q'
    option ifname 'eth0'
    option name 'vlan_eth0_2'
    option mtu '1500'
    option vid '2'

config device 'vlan_eth1_2'
        option type '8021q'
        option ifname 'eth1'
        option name 'vlan_eth1_2'
        option mtu '1500'
        option vid '2'

config device 'vlan_eth2_2'
        option type '8021q'
        option ifname 'eth2'
        option name 'vlan_eth2_2'
        option mtu '1500'
        option vid '2'

config device 'vlan_eth3_2'
        option type '8021q'
        option ifname 'eth3'
        option name 'vlan_eth3_2'
        option mtu '1500'
        option vid '2'

I think the issue is due to the bridge interface being only on vlan id 1 and having multicast snooping enabled. I tried many other changes, like bridging the bare eth0-3 interfaces, but most changes either locked me out of the switch configuration interface or made no difference.

Here is my /etc/config/network configuration file now. It looks a mess, but it works and since I didn’t want to make huge changes to the config and lock myself out (again,) it’s left like this. There is for sure a better way to do this, but these changes are good enough for me!

Recieving packets on pfSense now

Packets arriving on the right interfaces now. Only multicast packets for the relevant VLAN are captured, I believe because VMware is stripping the other VLANs from the Port Group.

CARP Packet Capture from pfSense working

tcpdump on the switch now

All CARP packets for all VLANs arriving at the switch port shown earlier. All CARP packets from all vlans are echoed on all ports.

# tcpdump -i eth1 -c 10 -n -e -T carp carp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
20:04:38.379521 00:00:5e:00:01:63 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 15, p 0, ethertype IPv4, 192.168.99.1 > 224.0.0.18: CARPv2-advertise 36: vhid=99 advbase=1 advskew=0 authlen=7 counter=4683731516847153633
20:04:38.379922 00:00:5e:00:01:05 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 2, p 0, ethertype IPv4, 192.168.5.1 > 224.0.0.18: CARPv2-advertise 36: vhid=5 advbase=1 advskew=0 authlen=7 counter=9456878584255914769
20:04:38.380439 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:04:38.380625 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 192.168.1.1 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 authlen=7 counter=13620309968023688989
20:04:38.380912 00:00:5e:00:01:0f > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 10.1.15.2 > 224.0.0.18: CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 authlen=7 counter=13306912573729581052
20:04:39.389533 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
20:04:39.389852 00:00:5e:00:01:05 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 2, p 0, ethertype IPv4, 192.168.5.1 > 224.0.0.18: CARPv2-advertise 36: vhid=5 advbase=1 advskew=0 authlen=7 counter=9456878584255914769
20:04:39.390086 00:00:5e:00:01:63 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 15, p 0, ethertype IPv4, 192.168.99.1 > 224.0.0.18: CARPv2-advertise 36: vhid=99 advbase=1 advskew=0 authlen=7 counter=4683731516847153633
20:04:39.390316 00:00:5e:00:01:0f > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 10.1.15.2 > 224.0.0.18: CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 authlen=7 counter=13306912573729581052
20:04:39.390533 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 192.168.1.1 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 authlen=7 counter=13620309968023688989

tcpdump on a linux endpoint now

tcpdump from a linux machine now shows carp packets from all VLANs on it’s physical port. Worth mentioning that the port is configured untagged on VLAN 15.

# tcpdump -i eth0 -c 10 -n -e -T carp carp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:15:14.122660 00:00:5e:00:01:63 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 192.168.99.1 > 224.0.0.18: CARPv2-advertise 36: vhid=99 advbase=1 advskew=0 authlen=7 counter=4683731516847153633
18:15:14.122965 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
18:15:14.123263 00:00:5e:00:01:05 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 2, p 0, ethertype IPv4, 192.168.5.1 > 224.0.0.18: CARPv2-advertise 36: vhid=5 advbase=1 advskew=0 authlen=7 counter=9456878584255914769
18:15:14.123316 00:00:5e:00:01:0f > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 10.1.15.2 > 224.0.0.18: CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 authlen=7 counter=13306912573729581052
18:15:14.123878 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 192.168.1.1 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 authlen=7 counter=13620309968023688989
18:15:15.134075 00:00:5e:00:01:05 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 2, p 0, ethertype IPv4, 192.168.5.1 > 224.0.0.18: CARPv2-advertise 36: vhid=5 advbase=1 advskew=0 authlen=7 counter=9456878584255914769
18:15:15.134351 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv4, 192.168.2.1 > 224.0.0.18: CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 authlen=7 counter=3809067134136217446
18:15:15.134627 00:00:5e:00:01:63 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 192.168.99.1 > 224.0.0.18: CARPv2-advertise 36: vhid=99 advbase=1 advskew=0 authlen=7 counter=4683731516847153633
18:15:15.134823 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 192.168.1.1 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 authlen=7 counter=13620309968023688989
18:15:15.134831 00:00:5e:00:01:0f > 01:00:5e:00:00:12, ethertype 802.1Q (0x8100), length 74: vlan 9, p 0, ethertype IPv4, 10.1.15.2 > 224.0.0.18: CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 authlen=7 counter=13306912573729581052
10 packets captured
10 packets received by filter
0 packets dropped by kernel
3 packets dropped by interface

Wireshark from windows inside a VMware virtual machine only shows carp packets on the vlan that it’s port group is configured to. Presumably VMware is stripping irrelevant VLANs. To show CARP traffic in Wireshark, select a VRRP packet, right click, click decode as and choose CARP from the list.

Wireshark Windows

The CARP interfaces all now show Master/Backup correctly. Failover is also working as expected.

Backup CARP

Written with StackEdit.

Saturday, 10 October 2020

RDP File Signing

RDP File Signing.md

Removing unknown publisher errors from RDP connection files

Ever seen this error or had it reported by a customer?

The publisher of this remote connection can’t be identified. Do you want to connect anyway?

With a publisher name of Unknown publisher

Unknown RDP Publisher

Getting rid of this is a 2 step process, first sign the RDP file, then set your clients to trust the signing certificate. Unfortunately this is only possible through a policy setting so the client machine needs to be domain joined or you must edit the local policy of the client.

Signing the RDP File

Signing the RDP file is done with the rdpsign tool, I found this built in on Server 2012, Windows 10 and Server 2019 so it looks like it’s there across the board. I don’t think the certificate requires anything special except for the Digital Signature key usage. I tested this with both a public wildcard certificate issued by Let’s Encrypt and with an AD Certificate Services issued “Code Signing” certificate. Presumably this would work with a self-signed certificate since the thumbprint is pinned later in group policy.

The /sha256 switch appears to be missing in earlier versions (2012, non-R2), but I used the /sha1 switch in it’s place and it accepted a sha256 certificate.

Run the command as follows

rdpsign.exe /sha256 <certificate thumbprint> filename.rdp

Example output from rdpsign

Once signed, if you open the .rdp file with notepad, you’ll see a signature block at the bottom of the file. Modifying any of the signscope elements of the file will cause the original unknown publisher warning to appear.

Example signed RDP file

Now that the file is signed, the first step is complete, but now there is just a different warning asking if you trust the publisher. The publisher listed is the subject name of the certificate used.

Example signed publisher warning

Setting the SHA1 hash in policy

To actually make the warning go away, the Specify SHA1 thumbprints of certificates representing trusted .rdp publishers setting in policy needs to be changed. In group policy the setting is as follows, the list is comma separated and there is an equivalent setting in the user policy.

Computer Configuration\Administrative Templates\Windows Components\Remote Desktop Services\Remote Desktop Connection Client

Here it is in a GPO

Policy location

Once the policy is set, update group policy on the client and reopen the .rdp file.

gpupdate output

And now there is no warning!

no warning

Written with StackEdit.

Sunday, 4 October 2020

Verifying RDP connections with Kerberos and Certificates

Verifying RDP connections with Kerberos and Certificates.md

Removing Certificate warnings for RDP

Certificate Warning

Certificate warnings on connection to an RDS server are not uncommon and are in fact normal when connecting to a non domain joined PC. They can be annoying, look unprofessional and can cause concern when users are required to connect.

Defaults & Self signed certificates

By default a non-domain joined PC will present a self-signed certificate when connecting. Since this isn’t trusted by the connecting client then a warning will be displayed.

When connected via RDP to a machine with a non trusted certificate, no security icon is shown in the connection bar.

no security icon on bar

While it’s possible to generate another self signed certificate with the DNS names you require, the certificate needs to be trusted by all client machines that connect otherwise the warning is displayed. Managing client’s trusted certificates is complex and not possible if you do not control the clients. In fact, it’s probably easier to just tick the ‘Don’t ask me again for connections to this computer’ box than it is to deploy a certificate to each client.

Ticking this box caches the certificate’s thumbprint in the REG_BINARY registry value, CertHash. The location in the registry is as follows:

HKEY_CURRENT_USER\SOFTWARE\Microsoft\Terminal Server Client\Servers\COMPUTERNAME

Registry Key Location

This is a per user setting so could be included in a login script for example. Here is some example PowerShell to set the value in the registry:

$thumbprint = "36558bf53757dd5c2ada081001323a969f576f4a"
$ComputerName = "commando"

$regPath = "HKCU:\SOFTWARE\Microsoft\Terminal Server Client\Servers\$($ComputerName)"
$thumbprintBinary = [byte[]] -split ($thumbprint -replace '..', '0x$& ')
New-ItemProperty -Path $regPath -Name CertHash -PropertyType Binary -Value $thumbprintBinary

Unfortunately, both methods of using self-signed certificates are cumbersome to manage.

Kerberos & Service Principal Names (SPNs)

By default you won’t get a certificate warning from a domain joined machine if connecting to it using it’s host name or fully qualified domain name (FQDN) since it will have an SPN registered for TERMSVC/hostname and TERMSVC/fqdn.

If you have a domain joined machine that you want to RDP to using an alternative name, you can use an SPN to allow Kerberos authentication to work. This only works for a single RDP endpoint since SPNs must be unique in the forest.

To create a new SPN, use the setspn utility

Show current SPNs

setspn -l computername

setspn list

Set a new SPN

setspn -s TERMSRV/aliasname computername

setspn set

Once a new SPN is added, connecting to the machine with the aliasname will show the connection is verified with Kerberos.

kerberos verified

Public Certificate Authority (CA) Signed certificate

It’s possible to use a wildcard, public CA signed certificate to secure an RDP connection. If you have a CA cert that provides the DNS name you need for connection then it’s possible to use this on all of the RDS servers behind a simple load balancer. To do this you must import the certificate in Windows. In my example I’m using a let’s encrypt wildcard certificate, the only requirement I can see is that it must have a greater than 2048 bit private key and include the “Server Authentication” Enhanced Key Usage.

Create a pfx bundle of your certificate on a machine with openssl installed. The following command includes the CA chain in the pfx.

openssl pkcs12 -export -out certificate.pfx -inkey privkey.pem -in cert.pem -certfile chain.pem

Once you have a pfx file you can import it in Windows. I imported to the default location, which is the local computer’s “personal” store. Right click on the pfx file and click import.

Import into Windows

Note that there is a private key available for the imported certificate

Imported certificate

Once imported, set the RDS certificate using PowerShell and WMI. WARNING: It’s worth mentioning that restarting the TermService service will kill current RDP connections so make sure to do this from the console of the machine. If the TermService service doesn’t find a valid certificate you could be locked out if you only have RDP access to the machine.

$GenSettingsPath = Get-WmiObject -Namespace "root\cimv2\TerminalServices" -Class "Win32_TSGeneralSetting"
Set-WmiInstance -Path $GenSettingsPath -Arguments @{SSLCertificateSHA1Hash="THUMBPRINT"}

Restart-Service TermService -Force

PowerShell output

Once connected, the connection is shown to be verified by a server certificate

certificate Verified image

Certificate from an Enterprise Active Directory (AD) CA

You can also secure an RDP connection to a single or group of machines using a digital certificate from your Enterprise, AD Certificate Authority. This is beneficial if you have a group of RDS servers behind a simple load balancer.

SECURITY WARNING: To generate a certificate from the Enterprise CA, we need to create a certificate template and publish in AD. Since we need arbitrary subject alternative names enabled in the template this is a dangerous template to create and leave enabled. For this example, I will create the template, publish it, request a certificate and then disable the template so it cannot be used automatically. This template could allow any domain computer to create a certificate for any name and therefore compromise the entire security of the CA. It would be best to secure the template so it requires CA manager approval before the certificate is issued. The following code snippets would need to be modified to handle a pending request.

Template creation steps

  1. On your enterprise CA, open the Certification Authority application
  2. Right click on Certificate Templates and click Manage

Click Manage

  1. The Certificate Templates Console opens, right click Computer and click Duplicate Template

Click Duplicate Template

  1. On the General tab, give the template an appropriate name, in this example I am using “RemoteDesktopComputer”

Name the template RemoteDesktopComputer

  1. Check the minimum key size is 2048-bits on the Cryptography tab

Crypto Tab

  1. Check that Server Authentication is enabled in the Application Polices section of the Extensions tab

Extensions Tab

  1. On the Subject Name tab, choose supply in the request. Note the security warning In a production environment, the “CA Certificate manager approval” option should be selected to ensure that certificates from this template are validated before issuance.

Subject Name Tab

  1. Review the Issuance Requirements tab, for this example the “CA Certificate manager approval” is unchecked, Do not do this in a production environment

Issuance Requirements Tab

  1. Click OK to save the template, close the Certificate Templates Console window
  2. In the Certification Authority window, Right click on Certificate Templates and click “Certificate Template to issue”

Click Certificate Template to issue

  1. Select the new template

Select the new template

Once you have a template created and published, the following PowerShell will request and issue a new certificate on the RDP server.

$CN = "CN=COMPUTER"
$dnsNames = @("COMPUTER", "computer.example.com", "loadbalancer.example.com")

$Cert = Get-Certificate -Template "RemoteDesktopComputer" `
	-SubjectName $CN -DnsName $dnsNames `
	-CertStoreLocation "cert:\LocalMachine\My" -Url ldap:

$Cert

At this point, check that the certificate in the computer certificates mmc is as expected and contains the correct DNS subject alternative names.

Issued Certificate

Once done, run the following in the same PowerShell session to apply the certificate. WARNING: It’s worth mentioning that restarting the TermService service will kill current RDP connections so make sure to do this from the console of the machine in case the certificate is invalid. If the TermService service doesn’t find a valid certificate you could be locked out if you only have RDP access to the machine.

if ($cert) {
	$GenSettingsPath = Get-WmiObject -Namespace "root\cimv2\TerminalServices" -Class "Win32_TSGeneralSetting"
	Set-WmiInstance -Path $GenSettingsPath -Arguments @{SSLCertificateSHA1Hash="$($Cert.Certificate.Thumbprint)"}

	Restart-Service TermService -Force
} else {
	Write-Host "Error generating certificate"
}

Once connected, the connection is shown to be verified by a server certificate

certificate Verified

IMPORTANT At this point, delete the published certificate template or secure it in another way

Delete the template

Written with StackEdit.