Sysadmins need money to replace aging servers
Multiple sysadmins report their server infrastructure is nearing end-of-life, and replacing it requires significant budget allocation. Some express frustration with the financial burden and suggest drastic measures to secure funding.
SOURCES (60)
“You joke but I happen to be a bit younger than Comic Chat! I'm just into retro tech, and I did administer IRC servers for friends for over a decade now where it occasionally comes up as a party trick. I even wrote a proxy…”
“Yeah CTO. Insight and Lenovo both dragging their feet made this happen and both are taking no responsibility. I would of never made the second order if they didn't tell me the first one would not be fulfilled.”
“Oh yeah for reference I have a 4TB that has a year under its belt at 5% loss. My 2TB i just deployed so no data.”
“When I first started at my current job we were running our public website in house. It was on a server with one raid-0 array.”
“While that's true, their lifespan met our normal lifecycle, so... [shrug]. I wouldn't do it on purpose, but I also wouldn't sweat it if we had to again.”
“I left a continuous ping to our head office server running over an international ISDN connection...for a long weekend. It racked up $38k in telco fees.... My stomach dropped when i got the monthly bill...but after some disputing the charges i got away with it....”
“I don't need help but I just wanted to tell you I love what you wrote brother. IT needs more helping hands and industry support, that's the only way we'll ever get people to become component.”
“Sorry that this happened to you guys. Unfortunately, smb's are the largest target because they are usually the most vulnerable due to lack of awareness, resources, or time to dedicate to a decent security program. Ill give you two resources to check out that you can leverage at this stage. The first is reaching out to your local CISA rep, the second is leveraging resources from the cyber readiness institute. CISA offers free training and resources a long with periodic assessments and health”
“That is... Awful. That's inexcusable lol. They could literally lay off 90% of HR and they'd have too many HR people”
“Many (many) years ago we had a failed disk in an array. MSP (because I was young and inexperienced) came out and replaced all disks in the array to rebuild from a backup. As they removed the dead disk, they gave me the good one as well. He told me that I should destroy the disks as he's rebuilding from backup. So I did as he suggested - pulled the screws, removed the lid breaking the vacuum, and removed the platters. I then went away on holiday for a week. Turns out the backup he was so cert”
“How/why are you running on-prem Exchange Server for such a small business? Tell me they’re not running Small Business Server. If so, they’re WAY past due for getting off of that. Getting them 365 subscriptions and migrating their mailboxes to the cloud seems like a no-brainer.”
“Copying and pasting from another comment - but I have installed over 30-35 servers all the way through in my life time, it has just been since 2018 since this was done. Recently I have been dealing with more LAN / Switches. Servers I do install - recently I just have been setting up and getting the other team to install software. I have experience in CALs. Again - just been a bit and more type this up before I even did some refreshing. This is something I am 110% confident I can install. Everyon”
“I agree and this is where I am just seeing options out there for current. My plan was server 2025, they use a terminal server now. I need to go back ( i was only there for a different job, but they told by at end of year they are looking to get it upgraded) - this was back in May. So now we are in the works of options and costs. Files are stored on-prem - 100%. honestly what they have now is trash. half their stuff is not backed up correctly, and i know that from my previous visit. From what I r”
“Select Shutdown instead of Restart on a server on the other side of the United States, where no one would be around to restart it for a few days.”
“Hey, I did something similar back when I knew nothing about sysadmin stuff and I uninstalled the SQL service from a server! Thankfully the DBs were still there, but everything had to be wired up again and it took a full day of client's sites down.”
“Y’all run your shit ass backwards I only run absolutely Windows only services on Windows if the service can be run on Linux it’s going on Linux. I like my uptime in years rather than hours. I literally have to bounce the DAG cluster on a daily basis to clear out logs.”
“Don't have a UPS but my system will power on when receiving power. Could that be used as a dumb trigger on one of these systems”
“the "(mostly)" after "keep the servers running" is doing a lot of work there. I assume the rest of the servers run on the goodwill of people arguing about pizza.”
“That's pretty quick in a corporate world. It took us a little over a year to get our servers.”
“I was going to attach his LinkdeIn post, but a copy/pasta will have to suffice. What are your thoughts? "I honestly wish I owned an MSP right now. There is no better time to be alive as an MSP. Get out there, spend time with your clients, help them cross the Al chasm, and bring janky business processes into the world of automation. And, if you're willing, we'd love to join you on that journey. Helping MSPs turn what they know about their clients into action is exactly why we're”
“backlog hell and being told to keep legacy gear limping along is now my normal working life. I've been told to keep some equipment going to well into the next decade, which will see me most of the way to retirement, and if they want to give me a package then that's just fine with me. in the meantime I have to balance a vendor that wants to offshore (read: enshittify) support for the product as step towards ending all support, virtually all of the hardware is under a LTB but manglement do”
“Lab equipment that does 1 specific niche job and the hospital that contracts my company refuse to replace due to the cost, uses the expansion port and serial ports with a custom made IO card Technically its not meant to use a C64 but the actual computer its meant to use is specific to this machine and I've never seen one, so years ago when there last one of those failed an engineer hacked a solution together with a C64”
“received the same …..broadcom does not give s fuck, unless your a high end datacenter….them buying vmware is one of the darkest days the IT industry ever had”
“Most compliance standards require updated systems. Trying to convince healthcare providers, PCI entities etc. that no they can’t run 2012 any more and it’s going to cost a lot to replace systems sees you pushed out of the quoting process. “We just want someone to manage what we have””
“I retire in 20 years so probs not 🤣🤣 will i be with current company also probs not, on another note I have recapped a few to get them going again”
“So I have a stock of c64 because theres 1 machine we support that uses them that would cost upwards of 50 mill to replace”
Clone it, upgrade it and if it breaks just run the old clone...
“I dont wanna be rude but your tech stack is massively out of date. This looks like an enterprise stack from 2010 (and I'm being generous here especially Java Swing)”
“That's not remotely accurate. Hdds are fine whether on always or powered on and off repeatedly, it's been tested many times by places with thousands more hdds than you will ever own in your life”
“Fully agree. It’s funny the homelab thing really seems to be a pretty good indicator of how much someone actually cares about tech. Not that id exclude someone just because they dont have a homelab, but for sure when I hear someone that I’m interviewing talk about stuff they’ve done or are doing in their homelab , that tells me they at least have some genuine interest in this area. Which I believe is essential to be successful in this field. Those who just took it for a job because they thought”
“are his asks leading to more revenue or customers? Lol no, only more tech debt and lost few long time clients.”
“Sorry for your loss, cancer sucks AI automation is where this field is headed, Sysadmin's will just be the human with the keys to production. Claude can do everything I can. From provisioning LUNs on the SAN to setting them up in your hypervisor and migration of VM's to it. Grab a Claude code CLI and build something you will be amazed. Example: 8 hours into a production outage(BACKEND, NOT customer facing). Dev's cant find the reason this legacy code doesn't work. Everyone kinda”
“your network needs a visit from a cyber security red team, and they they will see just how vulnerable you are... you need a security audit.”
“It sounds like they’re not fully utilizing their MSP; that’s the whole point of having redundancy when you’re unavailable for whatever reason lol. Unless that MSP doesn’t have techs of similar seniority level to you and they would’ve been spinning their wheels too. 🤣”
It doesn’t, it’s just my experience after 20 years in the field.
“I decided not to spend money on upgrading my old server and just use it to host my ZFS array for backups now. Only turned on when I want to do a backup. It's just a matter of choice at the end of the day. The same reason I don't purchase that next cool Linux device I like the look of - I spend enough time staring at screens and want to maintain some balance.”
“I've done this once and it was a while ago. Bring up new AD as a secondary and let it replicate. Promote secondary to primary and optionally, for testing, unplug the network from the old dc. I didn't see any issues with it unless you have some exotic stuff on the old one.”
“I'd automate everything and not tell anyone. Just use the free time for something else. Maybe a second job, maybe an OSS project. Maybe foe gaming.”
“I tried, some things worked, some didn't (802.1x), But on many things I feel like my hands are tied because I can't do much without his permission.”
“Yeah, it sound a little bad. I do not want to take it's side but the system is working ? Maybe that is what is tasked to do. Is it good? Absolutely not. Resistance to change in.l It is bad. As for the licenses, there are some idiots in other departments who will ask for that once a year.”
Wow. The grass is definitely greener on the other side. 🐐
“yes i know people are expecting things to get better but imo its never getting better and even if it did call it operational cost u still got use shit and build shit for months altho 15k is bit low”
“Get them to? They beg for it I borderline run a nonprofit cable company and they still bitch at me when it’s down”
“Agreed, it's never not going to look like you're not kicking someone while they're down. You should've made sure it was revoked when they offboarded”
“I would absolutely not touch anything at first. Yes, a backup is critical, but you're there to cover yourself and do your job, if previous failings mean there's no backup policy then it isn't on your head. Any software or hardware changes (even as little as increased disk thrashing during a backup) could kill something dead, if you're the one that caused it without a proper plan or approval, GG.”