Script randomly failing: if filesize doesn't change, send email

[Garacesh]

So, our CCTV management application sometimes falls over, seemingly randomly, and stops recording footage. Out comes the programmer in me.

 

Footage is archived off every 24 hours, so instead we look at the file sizes of the current folder, which stores data until it's archived off. Realistically, if the camera is recording no data, the filesize will not increase. If it is recording data, it should increase regularly (and does).

 

Code is as follows..

[color="#0000FF"]if[/color] ([color="#FF0000"]![/color]([color="#008080"][b]Test-Path[/b][/color] [color="#800000"]"D:\Crash Monitor\FileSizes.csv"[/color])) { [color="#008000"]# If the CSV for checking sizes doesn't exist[/color]
[color="#800080"]$Cameras[/color] = ([color="#800000"]"LIST"[/color],[color="#800000"] "OF"[/color], [color="#800000"]"MAC"[/color], [color="#800000"]"ADDRESSES"[/color], [color="#800000"]"FOR"[/color], [color="#800000"]"TEN"[/color], [color="#800000"]"HIGH"[/color], [color="#800000"]"TRAFFIC"[/color], [color="#800000"]"CCTV"[/color], [color="#800000"]"CAMERAS"[/color]) [color="#008000"]# File names on CCTV server are MacAddress_1, not friendly name.[/color]
[color="#0000FF"]foreach [/color]([color="#800080"]$Camer[/color]a [color="#0000FF"]in [/color][color="#800080"]$Cameras[/color]) {
	[color="#800080"]$CurrentCamera[/color] [color="#FF0000"]=[/color] [color="#008080"][b]New-Object[/b] [i]–TypeName[/i][/color] [color="#800000"]PSObject[/color]
	[color="#800080"]$CurrentCamera[/color] | [color="#008080"][b]Add-Member[/b] [i]–MemberType[/i][/color] [color="#800000"]NoteProperty[/color] [color="#008080"][i]–Name[/i][/color] [color="#800000"]MacAddress[/color] [color="#008080"][i]–Value[/i][/color] ($([color="#800080"]$Camera[/color]))
	[color="#800080"]$CurrentCamera[/color] | [color="#008080"][b]Add-Member[/b] [i]–MemberType[/i][/color] [color="#800000"]NoteProperty[/color] [color="#008080"][i]–Name[/i][/color] FileSize [color="#008080"][i]–Value[/i][/color] ($(([color="#008080"][b]Get-ChildItem[/b][/color] [color="#800000"]"D:\Recordings\$($Camera)_1"[/color] [color="#008080"][i]-Recurse[/i][/color] | [color="#008080"][b]Measure-Object[/b] [i]-Property[/i][/color] [color="#800000"]Length[/color] [color="#008080"][i]-Sum[/i][/color]).Sum)) [color="#008000"]# Filesize of cameras not-yet archived footage[/color]
	[color="#800080"]$CurrentCamera[/color] | [color="#008080"][b]Export-CSV[/b] [i]-Path[/i][/color] [color="#800000"]"D:\Crash Monitor\FileSizes.csv"[/color] [color="#008080"][i]-Append[/i][/color] [color="#008000"]# Create CSV for future checks[/color]
}
} [color="#0000FF"]else [/color]{
[color="#0000FF"]foreach [/color]([color="#800080"]$Camera[/color] [color="#0000FF"]in [/color]([color="#008080"]Import-Csv [i]-Path[/i][/color] [color="#800000"]"D:\Crash Monitor\FileSizes.csv"[/color])) {
	[color="#0000FF"]if [/color]([color="#800080"]$Camera[/color].FileSize [color="#FF0000"]-match[/color] (([color="#008080"][b]Get-ChildItem[/b][/color] [color="#800000"]"D:\Recordings\$($Camera.MacAddress)_1"[/color] | [color="#008080"][b]Measure-Object[/b] [i]-Property[/i][/color] [color="#800000"]Length[/color] [color="#008080"][i]-Sum[/i][/color]).Sum)) { [color="#008000"]# Compare csv record of filesize with current filesize[/color]
		[color="#800080"]$NoChange[/color] [color="#FF0000"]+=[/color] (($([color="#800080"]$Camera[/color].MacAddress)) + [color="#800000"]"`r`n"[/color])
		[color="#800080"]$Match[/color] [color="#FF0000"]++[/color]
	}		
	[color="#800080"]$CurrentCamera[/color] [color="#FF0000"]=[/color] [color="#008080"][b]New-Object[/b] [i]–TypeName[/i][/color][color="#800000"] PSObject[/color]
	[color="#800080"]$CurrentCamera[/color] | [color="#008080"][b]Add-Member[/b] [i]–MemberType[/i][/color] [color="#800000"]NoteProperty[/color] [color="#008080"][i]–Name[/i][/color] [color="#800000"]MacAddress [/color][color="#008080"][i]–Value[/i][/color] ([color="#800080"]$Camera[/color].MacAddress)
	[color="#800080"]$CurrentCamera[/color] | [color="#008080"][b]Add-Member[/b] [i]–MemberType[/i][/color] [color="#800000"]NoteProperty [/color][color="#008080"][i]–Name[/i][/color] [color="#800000"]FileSize [/color][color="#008080"][i]–Value[/i][/color] ($(([color="#008080"]Get-ChildItem [/color]($([color="#800000"]"D:\Recordings\$($Camera.MacAddress)_1"[/color]))[color="#008080"][i] -Recurse[/i][/color] | [color="#008080"][b]Measure-Object[/b] [i]-Property[/i][/color] [color="#800000"]Length[/color][color="#008080"][i] -Sum[/i][/color]).Sum)) [color="#008000"]# Filesize of cameras not-yet archived footage[/color]
	[color="#800080"]$CurrentCamera[/color] | [color="#008080"][b]Export-CSV[/b] [i]-Path[/i][/color] [color="#800000"]"D:\Crash Monitor\FileSizes2.csv"[/color] -Append [color="#008000"]# Create second CSV file[/color]
}
[color="#008080"][b]Move-Item[/b][/color] [color="#800000"]"D:\Crash Monitor\FileSizes2.csv" "D:\Crash Monitor\FileSizes.csv"[/color] [color="#008080"][i]-Force[/i][/color] [color="#008000"]# Overwrite old CSV file with new one[/color]
[color="#0000FF"]if [/color] ([color="#800080"]$Match[/color] [color="#FF0000"]-ge[/color] 7) {
	[color="#008080"][b]Send-MailMessage[/b][i] -smtpserver[/i][/color] [color="#800000"]'email-server'[/color] [color="#008080"][i]-To[/i][/color] [color="#800000"]"Garacesh "[/color][color="#008080"][i] -From[/i][/color] [color="#800000"]"Helpdesk "[/color] [i][color="#008080"]-Subject[/color][/i] ([color="#800000"]"Possible CCTV server failure: $Match/10"[/color]) [color="#008080"][i]-Body[/i][/color] [color="#800000"]"$Match/10 monitored cameras have not recorded any footage in the past 10 minutes. This may indicate a failure of the CCTV Management Application.`r`n`r`nThe MAC Addresses of the cameras are as follows:`r`n$NoChange"[/color]
}
}

That took way too long to colour-code.. I don't think I'll be doing that again in a hurry..

 

The problem is, it seems to just email me failures randomly - I can't discern any pattern at all. I've been monitoring the csv's it creates every 10 minutes and copying them off, just waiting for another failure, and this morning that failure arrived.

 

Filesizes at 08:40 vs 08:50, taken from the csv files of those times.

2501913872 / 2575314192

15674649872 / 16149130512

8932306192 / 9270471952

3810012432 / 3985648912

2116562192 / 2252877072

1285565712 / 1335373072

1694510352 / 1725967632

921185552 / 939535632

1282944272 / 1322265872

900214032 / 915942672

 

So, clearly every single folder has increased in size. Yet I still get an email that tells me the cameras haven't been recording. I've gone over my code again and again and I just can't find what my issue is..

 

Edit: Probably worth noting, the emails always say 10/10 cameras haven't increased in size.

Edited September 24, 2015 by Garacesh

[halbaradkenafin]

Have you tried replacing the Measure-Object section with either Select-Object -ExpandProperty Length or just wrapping the Get-ChildItem in parentheses and .length on the end? Could be some funky Measure-Object interaction that's screwing things up sometimes.

 

Other than that it looks fine and I can't see why else it would fail.

[Garacesh]

Select-Object -ExpandProperty Length or just wrapping the Get-ChildItem in parentheses and .length on the end?

 

I.. Hmm.

For some reason I didn't think of .Length.. I've used it for strings and arrays but this is my first time working with filesizes..

 

.Length works, so I'll put that in testing and see how it goes..

Bizarrely | Select-Object -ExpandProperty Length gives camera 6 and 7 a value of "System.Object[]" because the command ($(Get-ChildItem "D:\Recordings\$($Camera)_1" -Recurse | Select-Object -ExpandProperty Length)) returns the value twice. No idea why.

 

Re-signed and re-uploaded my code, let's see what happens..

 

Edit: Oh fruits. I just realised my if comparator didn't include -Recurse. That can't have been good.. Signed again. Uploaded again. Sigh.

Edited September 24, 2015 by Garacesh

[Garacesh]

Hm. It looks like it's using .Length to calculate how many files are in that folder, no the total file size. However, that number is rising, so each recording may to be its own file, so that might just be sufficient.

[halbaradkenafin]

Hm. It looks like it's using .Length to calculate how many files are in that folder, no the total file size. However, that number is rising, so each recording may to be its own file, so that might just be sufficient.

 

Ah, I misread your initial script and thought you were actually looking at the file sizes of a single file. Not sure why I assumed the files were saved without an extension but I've seen stranger things from some systems. If it's making files regularly enough then that way should work.

 

The next option would be to use the Measure-Object method but use -eq instead of -match, just in case the regex isn't being evaluated correctly. Or perhaps using the LastWriteTime property and ensuring it was within the last hour by doing something like:

 

if ((Get-ChildItem -Path C:\Files\Here | Sort-Object LastWriteTime -Descending | Select-Object -First 1).LastWriteTime -le (Get-Date).AddMinutes(-30))

[Garacesh]

Or perhaps using the LastWriteTime property

My initial thought was to use time, in a "Has this folder been written to in the past 10 minutes?" rather than having to bother with checking against a CSV file but the top-level folder (D:\Recordings\($($Camera.MacAddress)_1)) constantly has a Date Modified of when it was last archived off, no matter what time footage was last written there, because of the folder structure (Inside the top-level folder is a folder bearing the date of the recorded footage, and inside that is the actual footage)

 

.... But I didn't think to actually check. So now I feel like a bit of a boob.

It turns out that .LastWriteTime is in fact updated as files are written into subfolders. So you're right, I can rewrite it as "Is .LastWriteTime <10mins?" and cut out the CSV altogether.

[Garacesh]

Managed to finally get it sorted, and this seems to be working well.

 

$Match = 0
ForEach ($Camera in = ("LIST", "OF", "MAC", "ADDRESSES", "FOR", "TEN", "HIGH", "TRAFFIC", "CCTV", "CAMERAS") { # File names on CCTV server are MacAddress_1, not friendly name.
if ((((Get-ChildItem -Recurse "D:\Recordings\$($Camera)_1") | Sort-Object -Property LastWriteTime | Select-Object -Last 1).LastWriteTime) -lt ((Get-Date).AddMinutes(-10))) {
	$NoChange += ($Camera + "`r`n")
	$Match ++
}
}
if ($Match -ge 7) {
Send-MailMessage -smtpserver 'Email-Server' -To "Garacesh " -From "Helpdesk " -Subject ("Possible CCTV server failure: $Match/10") -Body "$Match/10 monitored cameras have not recorded any footage in the past 10 minutes. This may indicate a failure of the CCTV Management Application.`r`n`r`nThe MAC Addresses of the cameras are as follows:`r`n$NoChange"
}

 

Pull recursive list of each file, sort by lastwritetime (since there a lots of individual files), select the last one (newest as it's in ascending order).. That seems to work. Just using the folder they were contained in was causing false positives as the file itself didn't always update its date modified unless you accessed it.