IT mishaps you can come across thread

[BitMaster]

As some may know I service SoHo and smaller companies like up to ~25 people, nothing too big, so it's relative comparable to what can happen to you, me, everybody.

Today was such a day, nothing really special, replace old HW with Win10 with 6 new Win11 Pro clients which put together, GB-B850-9600x-1TB 990Pro-Seasonic Platinum 500w fanless-32GB_6000 and a DarkRock-5 in a nice and quiet bequet tower. The building was fun despite 1 MB was DOA, but they just flew. Did that 2 weeks ago and today I planned to roll them out. 

Save Data and Mail from old PC to server

Setup local LAN static IP and DNS

rename PC

join AD

setup accounts

install all LAN printers

restore Data and Mail

rinse & repeat...1..2..3..4..5..BANG !!!!!

I accidently ended up on the wrong desktop, i was in RDP on Server instead of on the local Desktop, both had 2 File Explorers open, Task manager, windows over windows,   I opened the the RENAME THIS PC on the wrong Desktop !! ME IDIOT !!!!

 

I RENAMED THE Active Directory Controller, AD-DC, to PC-3 !!!!!!!!!!!!!!!!!!!!!!!! hit OK and then found out  FUUUUUuuuuuuu§&   WRONG RDP --> WRONG FILE EXPLORER --> WRONG CONTROL PANEL -----> GOD DAMN WRONG MACHINE to rename     and I picked the worst machine you can possibly rename while in production and running  THE DOMAIN CONTROLLER     

My blood froze, my pulse went to 180 from chilling low 60s, my promise to be home at 19:30h THE LATEST suddenly became a lie and my

biggest fear was: "You're gonna either have some damn luck, PRETTY SOON, or you end up with Acronis, putting the Backup from last night on"

Ya, after you made the bootstick, 20min, went upstairs to the server. a real ClusterFu%§ visualising what was ahead of me if I didnt come up with

something quick, quickly .

So, I know how to rename a DC, you DON'T do it like you do with any other Windows machine, it's a Powershell command, 3 of them iirc, with reboots and DNS entries. Not complicated, but must be obeyed or you screw up the directory, or your access to it...f you only run 1 DC, that must be said to be fair.

OK..Google...   it says...you did the WRONG thing, there is NO easy way out, you either use a Backup or REINSTITUTE the Domain, which basically means you might as well reinstall the whole damn thing, that's 1 day alone if ALL goes right. I did that server and it is not difficult, just a lot of stuff and it takes time and care.

So...I had no choice.. I had to reboot it while I was there, no way out, you dont want to expore the outcome of that 40km away and only VPN,Teamviewer/RDP/VNC at hand.

OK, the server has an Enterprise Dell iDrac that I can access from here, that would have worked even from remote...but here it goes.

So I rebooted, there was no "rename again" back to SRV possible, it needed the reboot to commit. So I did.

It booted to login screen, so far so good. At least no AD DB error while booting, that is a total mess then. OK, OK...so far great...try  to login..

Login as admin and pwd --> Red Pop-Up --> No security entry in AD Database for this workstation. Login denied!     OHH SH!T

You cannot log into the local domain aka PC/Server on a DC, it is ALWAYS a log in into AD-admin. So I could not do    SRV\administrator to log into

the Server locally w/o Domain. I tried despite I knew it wouldn't work and it didn't, waah ! 

 

You're finally screwed !!!!

You have a booting 2022 AD-Controller with a wrong name that doesnt connect to the network, doesnt show shares you could connect to and

SOMEHOW work, heck, if the SQL's would run, I'd have more time to sort out things...but this beast doesnt do ANYTHING but boot to Welcome

screen and sit there forever. mI also couldn't even finish the 6th PC and drive home, I was about to join AD when that happened

 

There is no guide on Google or in AI that tells you how to redo that mishap. It's a nail in the server's coffin that you usually don't get out again.

That is what gave me headaches, the not having options to fix it, just FIX IT, god damn, it's only a few lines of code...ok maybe a few more it's AD

and thus deeply DNS integrated...WTF, how could you fall into exactly this pit, this No-Way-Out pit, with a big "How to Sink a DC" sign above it in

Neon. Aaarghhh, a big client..and you screwed up his DC, it better runs Monday morning 7am, no matter what, no matter.

 

So...I have it all on Acronis on a Linux Server as well as in Acronis Cloud, that would be the ultimate thing to fix it if I didnt come up with a solution,

something dead simple, stupid simple, as quick&dirty as the renaming was.  But what.

OK,...lets try F8 and Safe Mode, maybe there is something that can be done.....but wait....I cannot reboot or shut it down.

The DC-Server Welcome Screen in contrast to Client Windows OS has no Button to restart or shutdown, it's a safe guard. 

Well, lets see Dell's iDrac....but there have never been drivers installed in the OS for any iDrac, so I did not expect see a graceful OS shutdown/reboot option, just the normal AC operations which I knew. I was right, no graceful options, warmstart or cold restart or OFF.   Yeah, what a freaking nightmare, I have to reset the server now as well, a Russian Roulette with AD Database and RAID's, you never reset server unless you really really really have to. I had to ! 

Show balls !!  Grab it btP !

I hit warm restart and was then greatet with a UEFI Secure Boot error, luckily I could just say "screw you, I already have enough trouble!" and clicked it away and it kept booting through the many Dell UEFI and FW screens, I kept hitting F8 over and over agin.

Made it, Boot options came up  1 to 9..or 0  god knows.

4 is Safe Mode.... but wait....boot to LAST KNOWN WORKING CONFIGURATION ! ????   

OK, that is not ment to fix any AD structural damage if there is any but it should rewind the actual PC-settings I changed a reboot ago !?

Let's try, there is really nothing to loose but the 5min it takes to boot. If it screws up and crashes I go the Acronis route, chill and wait until restored,

if Acronis lets me down I am screwed, then I am gonna spend Sunday here, for free, fixing my sh!T. Monday 7am is deadline.

 

The server booted, as before, login available, no errors so far....now try the usual   DOMAIN\administrator  PWD and see if it locks up.

TATTAA....It fixed it, dead simple decade old rescue option fixed the misconfig, in a very lucky way I admit. The narrow path with too much domain

on one side and too severly misconfigured HW on the other side, this option was golden, it saved a lot of work, money and effort.

 

And I, I will, after 30 years, watch even closer now..on which RDP or Desktop I am before I rename a machine in a Directory !

I really felt baaaad the first 5 minutes after it happened, the"You screwed it Up!" hit hard.

 

Anyway, that server booted as nothing ever had happened, DNS was correct, DHCP changes I made were also still present, all good.

Fixed that last PC, printers, users, Mail, done.

 

When I got home, 2h late, my step daughter looked at me, said  "You know how late you are!!?" went out and took off...LoL

I still felt great, still do, having a beer and typing this.... better than installing a 2022 SRV with AD, Rolls, Software, Users and what not else till 2am

and all Sunday too.

Done that, thank you, my beer tastes a lot better than that !   

 

 

God save the command "Revert to last known working configuration"

Prost 

 

 

 

Edited 22 hours ago by BitMaster