Restart Devices on a Schedule

dyoder
dyoder Member Posts: 52 ✭✭✭
edited December 2023 in Remote Monitoring

I have what I believe to be a very simple task:

I need to restart a few endpoints on a regular schedule every night, so each day those endpoints are updated and ready to go without needing any restarts during working hours.

This is not extreme, nor is it unreasonable - nonetheless, this cannot be done using Atera. If there is someone out there that has a solution, I'm all ears. But I'm currently very tired of having to script away all the road blocks Atera affords me.

This post is really for product development, but maybe someone else will find it entertaining.

Let's Begin

Before Configuration Policies existed, my only option to acheive this task was to use IT Automation. So I created an automation policy called "Reboot everyday at 3am". The only tasks selected in this IT Automation are:

  1. The schedule to run every day at 3am
  2. Create a System Restore point
  3. Reboot
2023-09-06 12_08_50-Atera - RMM software _ PSA & Remote Access for MSPs - Brave.png

I applied this IT Automation to the endpoints that needed it - these are non-servers at multiple customers that typically stay on 24x7 (like conference room computers). Sometimes they get shut off in the course of being used, but it isn't a problem.

The Problem

I've received complaints over the last few months of endpoints restarting at random intervals, without warning, and during working hours.

I opened a support ticket and the technician determined it was caused by something I did. At the time it wasn't very impactful and I figured maybe it was an isolated incident, so I didn't dig too much into it. And for a while, things were silent on this topic.

This morning I get a call from one of the Directors at my client. They had a Teams meeting in the conference room that lasted 1.5hrs, and during that time the endpoint restarted 3 times during their meeting. I knew I needed to get to the bottom of this, so I opened another support ticket.

The technician found all 3 restarts in the Windows Event logs, and the custom comment on the restarts indicated they were triggered by Atera.

Use a Configuration Policy!

I detailed what I had configured in Atera and the technician did a little digging. He found the culprit - all IT Automation policies are set to queue up missed policy applications for offline agents for up to 1 week.

2023-09-06 12_21_23-Atera - RMM software _ PSA & Remote Access for MSPs - Brave.png

This means that particular endpoint was offline during 3 scheduled applications of the IT Automation policy "Reboot everyday at 3am". Now that it's back online, that policy ran 3 times consecutively.

The technician proposes a new solution - use a Configuration Policy. This way I would be able to control the ability for the endpoint to reboot. I thought, great!

I had setup a test Configuration Policy about 5 months ago to test out reboot notifications that would be displayed to the user. I applied the test policy to my laptop and forgot about it. In 5 months time I had never received any notification that my laptop needed to restart. So in the back of my mind, I had categorized the Configuration Policy as a beta feature and did nothing further with it. I mentioned this to the technician and he did some more digging.

He then discovered the issue here - I needed to have the option called "Reboot when needed" checked in the IT Automation policy. The only place this is detailed in the product documentation is here: https://support.atera.com/hc/en-us/articles/5499183257884

2023-09-06 12_33_54-Configuration policies – Atera Support - Brave.png

It is not covered in the Troubleshooting section at the bottom of that article, nor is it covered in the guide on setting up a Configuration Policy: https://support.atera.com/hc/en-us/articles/5499199743388-Set-up-configuration-policies

No worries, I didn't read it all the way through - that's on me. But then I started thinking about the setting itself "Reboot when needed". And I thought, but I don't want to reboot when needed - I want to guarantee the endpoints get rebooted at 3am every day…

An Impossible Solution

My constraints on the original task are as follows:

  1. The endpoint must reboot at 3am every day.
  2. The endpoint must not reboot outside of this schedule, especially during working hours.

Again, not extreme and not unreasonable.

As it turns out, this cannot be done. If I specify the "Reboot" option in the IT Automation policy, the endpoint will reboot regardless of any Configuration Policy.

2023-09-06 12_55_08-Atera - RMM software _ PSA & Remote Access for MSPs - Brave.png

And if I specify the "Reboot when needed" option in the IT Automation policy, the endpoint will only be rebooted if Windows Update requests a reboot.

2023-09-06 12_56_54-Atera - RMM software _ PSA & Remote Access for MSPs - Brave.png

And the Configuration Policy cannot be used to initiate a reboot, period. It can only be used to control reboots as a result of Windows Update.

I thought, ok - the only solution is to stop all IT Automation policies from running missed applications for offline endpoints…

2023-09-06 12_59_25-Atera - RMM software _ PSA & Remote Access for MSPs - Brave.png

Nope! I can't do that either.

Ok ok ok, last resort - I can script the reboot. But then the only way I can apply the script on a schedule is to use an IT Automation policy, and that brings us right back to square 1 🤦‍♂️

A Solution, Anybody…

At this point I have a headache.

I could create a script based threshold that checks to see if the computer has not been restarted in 24hrs; if not, raise an alert and run a script automatically to reboot the endpoint. But I cannot schedule this option, so this is no good.

I could create a script that runs at 3am and checks the Windows Event log to see when it was last restarted; if it's been more than 24hrs, reboot. But if the endpoint has been turned off for more than 24hrs this will reboot one time when the IT Automation policy applies. So this isn't a good solution either.

I could create a script exactly like above, but this one queries WMI for the boot time. Then I need to diff the boot time from the current time to see if the delta is >24hrs; if so, reboot. Well, this one may actually work.

Final Thoughts

Is this a very specific use case? Yes. Is this entirely annecdotal? Yes. Should this be an issue at all? Absolutely not.

Here are some things that would improve the functionality of Atera and avoid this and probably other problems I haven't encountered yet:

  1. Add an option to not queue up missed IT Automation policies at all - if they're missed, they're missed.
  2. Add an option to not stack all missed IT Automation policies and run them at once. I'm not sure what the use case is for the current configuration, but allowing me to configure running a missed policy once would be nice.
  3. Allow each IT Automation policy have a configurable setting for running on offline agents, instead of the current global setting.
  4. Since Configuration Policies seem to be designed to control the state of an endpoint, add a state in there to limit uptime (reboot).

Tagged:

Comments

  • yasminproduct16
    yasminproduct16 Internal Posts: 16 ✭

    Hi, thank you very much for this feedback.
    I understand your frustration and we'll take this all into consideration.
    We will also work to improve the documentation and hopefully clarify it.
    We're also working on a feature that will allow you to schedule a future reboot so hopefully that might help. The MVP won't include recurring tasks but you can schedule multiple reboots, which we may consider down the line.

  • frank.pietersma
    frank.pietersma Member Posts: 78 ✭✭✭

    What could help if you could choose per automation policy if it should run on offline agents.
    I have always find it strange that this option counts for ALL automation policies,

  • dfletcher
    dfletcher Member Posts: 30 ✭✭
    https://community.atera.com/discussion/comment/840#Comment_840

    This. 100%.

  • timothy.walker
    timothy.walker Member Posts: 1 ✭

    I find it really frustrating that the 'Run on offline agents' is a global setting and does not allow an individual IT automation to override it.

    I too have some IT automations that I only want to run at specific times and if the machine is off,. skip it and others where I want it queued because they're only scheduled weekly and I need them to run when the computer comes online….

    As far as the original problem, my solution was to create a script that sets a windows schedule to restart at a specific time. It doesn't get logged in Atera but at least it runs when it's meant too…

  • DP
    DP Member Posts: 56 ✭✭✭

    Thanks for documenting this @dyoder - I mentioned this to support long ago when explaining that the update approval process didn't run as expected - they "almost" have this fixed.

    The fix I also recommended was to allow the policy to have a local override for "Run on offline agents" . They agreed, but didn't have a solution yet.

  • DP
    DP Member Posts: 56 ✭✭✭

    I had this reboot issue way back when I started with Atera. The configuration policy is only triggered when the Windows updates are installed TO trigger it. I have machines using Atera toast notications to ask for a restart every 5 hours (which seems to work ok). On machines I needed to restart at a certain time I created 2 scripts. They worked OK - but needed improvement.


    _Check, Restart, if NO user logged in and another with a countdown that could be cancelled.

    $reboot_required = (Get-WmiObject -Class Win32_OperatingSystem).RebootRequired
    
$logged_in = $false
Check if any user is logged in
$users = Get-WmiObject -Class Win32_ComputerSystem | Select-Object -ExpandProperty UserName
    
foreach ($user in $users) {
    
if ($user -ne $null) {
    
$logged_in = $true
    
break
    
}
    
}
Check registry for additional reboot requirements
$registry_reboot_required = $false
    
$registry_reboot_required = $registry_reboot_required -or (Test-Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Services\Pending*")
    
$registry_reboot_required = $registry_reboot_required -or (Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager" -Name "PendingFileRenameOperations" -ErrorAction SilentlyContinue) -ne $null
    
$registry_reboot_required = $registry_reboot_required -or (Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager" -Name "PendingFileRenameOperations2" -ErrorAction SilentlyContinue) -ne $null
    
$registry_reboot_required = $registry_reboot_required -or (Test-Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\RebootRequired")
    
$registry_reboot_required = $registry_reboot_required -or (Test-Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\PostRebootReporting")
    
$registry_reboot_required = $registry_reboot_required -or (Test-Path "HKLM:\Software\Microsoft\Windows\CurrentVersion\Component Based Servicing\RebootPending")
    
$registry_reboot_required = $registry_reboot_required -or (Test-Path "HKLM:\Software\Microsoft\Windows\CurrentVersion\Component Based Servicing\RebootInProgress")
    
$registry_reboot_required = $registry_reboot_required -or (Test-Path "HKLM:\Software\Microsoft\Windows\CurrentVersion\Component Based Servicing\PackagesPending")
    
$registry_reboot_required = $registry_reboot_required -or (Test-Path "HKLM:\SOFTWARE\Microsoft\ServerManager\CurrentRebootAttempts")
if (($reboot_required -or $registry_reboot_required) -and !$logged_in) {
    
Write-Host "Restarting computer..."
    
Restart-Computer -Force
    
exit 0 # Success
    
} elseif ($reboot_required -or $registry_reboot_required) {
    
Write-Host "Restart required, but user is logged in."
    
exit 1 # User logged in
    
} else {
    
Write-Host "No restart required."
    
exit 2 # No restart required
    
}

    This method checks the same places that Atera checks for when it says "reboot pending" in the console - If a user is logged in, it will not restart (mostly) - I'm not sure if it also checks RDP logins, etc.

    Perhaps this could help - I'm sure you could make it sing.

Topics