Oh, where to begin? Like most of the IT industry and home lab enthusiasts I was drawn in by ScreenConnect the now defunct remote software company that was bought out by ConnectWise and renamed Control. When I first came across this server it was initially configured on a Ubuntu 14 desktop environment.
I wanted this to become a production system and with the delicate Mono config decided to try my luck converting it. This caused me some grief but I did not know what I know now about Ubuntu or even Linux in general in 2014. After removing the gnome desktop environment it lived like this for a few years with problem after problem that was slowly patched and fixed.
Enter connectwise. Connectwise bought SC and turned it into Control. This was of no moment at the time, using NGINX as a reverse proxy I had SSL and a manageable interface as Mono; Screenconnects built in web engine slowly faded into EOL status and no longer supported modern browsers.
It was time for 19.04 LTS and after attempting upgrade it failed. Not only was I stuck on a Gen1 VM I could no longer update the core OS from 18.04 without destroying SC. So with Connectwise predatory pricing scheme, and my lifetime SC license no longer activating on their servers I could do nothing but wait.
So I did, With 20.04 coming out I wanted to try again. I was finished the migration to encrypted and protected VMs with a slew of other security related modifications the only server out was Screenconnect.
I did the upgrade from 18.04 to 20.04 and as anticipated it broke. This time however I wanted to fix it. First and foremost I saw that the service was starting, but only After I was able to console into the server. SSH was not working. I figured if SSH did not fuction it stood to reason this may be firewall related. After a quick check:
Sudo ufw status
I was confused to find it disabled. After enabling the firewall
sudo ufw start
sudo ufw enable
This completed without issue, and a check of my rules showed they were still in place. I decided at this point to reboot, hoping that maybe this was always the problem. After however, I was greeted with the same problem. This needed to be fixed first for me to proceed. I need the firewall to start with the system. What was odd was that all of the enable commands via systemctl or otherwise did not report an error.
After some digging I managed to find my issue.
sudo nano /lib/systemd/system/ufw.service
Had a line in it regarding firewall start metrics.
Before=network.target
This made no sense, we need the firewall to start AFTER the network comes up so it does not fail and stop.
After=network.target
Done. After yet another reboot the firewall service was working correctly. Now it was time to check on Screenconnect. While installed and intact after the upgrade still was not starting. Thankfully SC is configured to log its own errors.
at ScreenConnect.Program.Main (System.String[] args) [0x00000] in :0
[ERROR] FATAL UNHANDLED EXCEPTION: System.TypeInitializationException: An exception was thrown by the type initializer for libc ---> System.DllNotFoundException: libdl.so
at (wrapper managed-to-native) ScreenConnect.MonoNative+libdlProxy:dlopenPlatformInvoke (void,int) at ScreenConnect.MonoNative+libdlProxy.dlopen (System.Void , Int32 ) [0x00000] in :0
at ScreenConnect.MonoToolkit+MonoDiskNativeLibrary.TryLoadNativeLibrary (System.String libraryPath) [0x00000] in :0
at ScreenConnect.DiskNativeLibrary..ctor (System.String libraryPath) [0x00000] in :0
at ScreenConnect.MonoToolkit+MonoDiskNativeLibrary..ctor (System.String libraryPath) [0x00000] in :0
at ScreenConnect.MonoToolkit.LoadNativeLibraryFromDisk (System.String libraryPath) [0x00000] in :0
at ScreenConnect.NativeLibrary.LoadLibrary (System.String libraryName, System.Type lookInAssemblyWithType) [0x00000] in :0
at ScreenConnect.NativeLibrary.LoadLibrary (System.Type type, System.Type lookInAssemblyWithType) [0x00000] in :0
at ScreenConnect.NativeLibrary.LoadProxy (System.Type type, System.Type lookInAssemblyWithType) [0x00000] in :0
at ScreenConnect.NativeLibrary.LoadProxy[libc] (System.Type lookInAssemblyWithType) [0x00000] in :0
at ScreenConnect.MonoNative+libc..cctor () [0x00000] in :0
--- End of inner exception stack trace ---
Rough. IT looks like it was having some issues with libc, libdl after a quick look around using find I came across broken symlinks in /lib/x86_64-linux-gnu where the files resided. It looked like glibc was updated and the existing links were broken.
SSH made this easy because the broken links showed up as bright red. Time to fix them.
sudo ln -s libc-2.31.so libc.so
sudo ln -s libdl-2.31.so libdl.so
sudo ln -s libz.so.1.2.11 libz.so
After the re-link a reboot for good measure and WE WERE BACK!
20.04 running well and screenconnect letting me touch my machines.
Life is good.
BONUS: If you want colors in your sessions
sudo nano ~/.bashrc
Then uncomment the line
force_color_prompt=yes