Answered by:
Monthly Report Error?

Question
-
Just looked at a monthly report (I've had 2.0 installed a week) and this is what I see.
Potential Threats Found This Month: 198 ~ I've received one notification of a potential threat and it was today. Are there other things included on this?
Performance Plus
Tune-Up Not yet performed. Yet if I look at the status it is good and it states the last tune-up was performed on 7/24 (which it was)
The backup date it also off. I'm guessing the report is not real-time?
Thursday, July 26, 2007 7:51 PM
Answers
-
Taking my theory here a step further, I'm guessing that the reason these potential threats were being included was to attempt to show the skeptics that OneCare is really protecting them. We've heard the complaints from a few that they haven't seen anything actually happen, so they're wondering if OneCare is really doing anything.
The funny part is that if OneCare is really doing its job, the typical user really shouldn't ever see more than an occaional firewall alert about something new accessing the Internet. Only those who tempt fate by opening questionable email attachments or browsing risky web sites should even see an AntiMalware alert in most cases.
This is the paradox of truly good protection, it's not noisy unless it has to be. Unlike the antispyware that alert every time they find a 'bad' cookie, OneCare reserves this for real issues that deserve the user's attention.
Finding this balance is the difficult part and the Monthly Report looks to me like the Onecare attempt. It tries to inform the user what OneCare has been doing without inflamed scare tactics or spurious alerts. How to do this without adding the very complexity it's designed to avoid is the deeper issue though.
I've experienced exactly the kind of confusing graphs and sketchy explanations we've seen so far in the sophisticated vulnerability scanner reports of the several products I've worked with. That's why we can make good money explaining these reports to customers, the information is right there in front of your face, but totally obscured at the same time.
I believe the flaw is inherent in the attempt to 'quantify', which is a management (read 'Bean Counter') approach, rather than really trying to explain what's happening. In this case, the offending 'number' appears to be nothing more than a count of "We don't know what these are yet" items that would be better off ignored [by the user] until they're identified. I haven't really tried to analyze any of the other entries yet, since I truly have almost no interest in this information in the first place.
I don't know what might really be of interest or value to the non-technical user, but I'm almost sure that these numbers aren't the answer. More likely, they'd care whether all of the most recent viruses displayed in the news had been added to detections and an even more useless count of how many new malware detections have been added in the last week/month. Again, I don't really care about any of this myself, but it would depend what the purpose behind the report is supposed to be in the first place.
All of the information presented is simply a summary of existing entries in the Support Log, or more accurately in the OneCare database. Which of the items there actually generate interest would be valuable to know. I personally only care that the product blocks malware, either automatically, via reduced vulnerability or by requesting my input when the potential threat is unknown.
OneCareBear
- Marked as answer by Stephen BootsMVP, Moderator Wednesday, December 24, 2008 1:40 PM
Monday, August 27, 2007 4:31 AMModerator
All replies
-
I think the report is a little wacky, myself. :-)
-steve
Friday, July 27, 2007 1:34 AMModerator -
Same here What's up With that huh ?
Friday, July 27, 2007 7:24 PM -
Don't know...my report also shows Tune-up not yet run, by the way.
-steve
Friday, July 27, 2007 11:09 PMModerator -
My latest report shows
- Potential threats found this month 1278 (would have been nice to know what they were!)
-
Total potential threats found to date 3 (so what happened to the other 1275 mentioned above lol!)
-
Number of updates installed each week - Jul 12 (15), Jul 19 (15) & Jul 26 (0) - so what happened on the 26th?
-
Number of system scans = 0 - but I noticed 3!
It would be nice to know what the threats were and what was done with them (I did find 2 items in quarantine)?
Tom
Saturday, August 25, 2007 1:46 PM -
Just to let you know, we have asked the OneCare team for an explanation of the monthly report in some detail as we are all seeing these oddities in the report.
-steve
Sunday, August 26, 2007 2:10 AMModerator -
Is anybody bothering to look in their Support Logs? It's quite clear where these numbers are coming from, but less clear what the items generating them might be in some cases.
In general, they are detected processes that are generating 'ANTIVIRUS_ONDEMAND' scans to be performed. Most likely they are unidentified processes that the active mode monitoring processes from Defender have detected and then passed to the AV On Demand process to scan.
Most of these in my case are only identified by a pid (Process ID) number, but at least one is directly identified as related to a 'reconfigurationDataStore.xml' operating on a like named .dat file located in the VMware Workstation Program Files folder.
The fact that the number of these entries is dropping and no new entries have been logged on my PC since August 15th leads me to believe that most have been identified as non-threats and are no longer being detected as 'potential threats'.
This all seems reasonable to me. If you continue to complain about this entry though, I'm quite certain they'll take this input from the 'experts' who are always asking for more control and visibility and simply remove all indications of this from the reports.
OneCarebear
Sunday, August 26, 2007 7:59 PMModerator -
Onecarebear,
Whilst I do not doubt that what you say is correct, Onecare is aimed at, I thought, the home user who just wants to set and forget. So if they are presented with a report with results similar to the one I received they are going to be concerned (and will not have the knowledge/desire to go through various support logs).
I believe is that all people are asking is what has been detected as suspect and what has been done with it? It's not unknown for an antivirus/spyware program to falsly detect a threat and delte it.
Tom
Sunday, August 26, 2007 8:15 PM -
Yup, and the fact that this group can't understand what such a simple message is telling them probably means it's too much for the less knowledgeable users to even see. For that reason, you've just confirmed what I was saying and I suspect this entry will disappear totally in future versions.
This is what's so interesting about such attempts at including 'expert' information in even summary reports and exactly parallel to why 'Advanced' sections of configuration don't work for foolproof programs. It very quickly becomes obvious that even knowledgeable users are overwhelmed by too much [simple] information.
Since the information that they were attempting to display in this case is really too technical for all but the most knowledgeable PC user, generally with a least a small amount of process level programming background, it's better to simply leave this information buried in the Support Logs where it belongs. Those who really want to know such things will find them and no one else really gives a .....
OneCareBear
Sunday, August 26, 2007 8:33 PMModerator -
LOL! OneCareBear, no, I never checked the support log to compare it to the monthly report. I guess I agree that the entry of potential threats really should not be reported.
-steve
Monday, August 27, 2007 1:07 AMModerator -
Taking my theory here a step further, I'm guessing that the reason these potential threats were being included was to attempt to show the skeptics that OneCare is really protecting them. We've heard the complaints from a few that they haven't seen anything actually happen, so they're wondering if OneCare is really doing anything.
The funny part is that if OneCare is really doing its job, the typical user really shouldn't ever see more than an occaional firewall alert about something new accessing the Internet. Only those who tempt fate by opening questionable email attachments or browsing risky web sites should even see an AntiMalware alert in most cases.
This is the paradox of truly good protection, it's not noisy unless it has to be. Unlike the antispyware that alert every time they find a 'bad' cookie, OneCare reserves this for real issues that deserve the user's attention.
Finding this balance is the difficult part and the Monthly Report looks to me like the Onecare attempt. It tries to inform the user what OneCare has been doing without inflamed scare tactics or spurious alerts. How to do this without adding the very complexity it's designed to avoid is the deeper issue though.
I've experienced exactly the kind of confusing graphs and sketchy explanations we've seen so far in the sophisticated vulnerability scanner reports of the several products I've worked with. That's why we can make good money explaining these reports to customers, the information is right there in front of your face, but totally obscured at the same time.
I believe the flaw is inherent in the attempt to 'quantify', which is a management (read 'Bean Counter') approach, rather than really trying to explain what's happening. In this case, the offending 'number' appears to be nothing more than a count of "We don't know what these are yet" items that would be better off ignored [by the user] until they're identified. I haven't really tried to analyze any of the other entries yet, since I truly have almost no interest in this information in the first place.
I don't know what might really be of interest or value to the non-technical user, but I'm almost sure that these numbers aren't the answer. More likely, they'd care whether all of the most recent viruses displayed in the news had been added to detections and an even more useless count of how many new malware detections have been added in the last week/month. Again, I don't really care about any of this myself, but it would depend what the purpose behind the report is supposed to be in the first place.
All of the information presented is simply a summary of existing entries in the Support Log, or more accurately in the OneCare database. Which of the items there actually generate interest would be valuable to know. I personally only care that the product blocks malware, either automatically, via reduced vulnerability or by requesting my input when the potential threat is unknown.
OneCareBear
- Marked as answer by Stephen BootsMVP, Moderator Wednesday, December 24, 2008 1:40 PM
Monday, August 27, 2007 4:31 AMModerator -
Anyone else have any other preferences for what should be included in a 'Monthly Report' or any reporting for that matter?
As I stated, I'm really relatively ambivalent about most of these items since I have my own methods for measuring the effectiveness of my antimalware suite. I do see why some might want more feedback though and other than the specific item of 'potential threats' I have no strong feelings either way. The other numbers listed relating to actual detected or removed items are probably more useful, though I'm still not sure how well they'll be understood.
My own measurement is to perform simple tests online with things like Eicar and Shields Up! to confirm basic function. After that I simply have to trust that the OneCare and AntiMalware Teams responsible for the program and detections are doing their job, as with any protection suite I might purchase.
This is the time to say what you like or would rather see included if possible and how it should be presented. I personally have a somewhat dim view of numbers and graphs, since they haven't worked well with the Vulnerability scanners I've seen. However, the linked explanations of each vulnerability and other such information are generally useful. In OneCare terms this would include things like links to any actual detections found (Quarantine Summary) and other such specific information that might exist in the database.
Though this information is actually somewhat technical, guiding the non-technical user to online descriptions of the items detected and/or removed would actually have more real substance and potential for learning for those who wanted to know more. Displaying the counters is fine, but as the potential threats example showed, they're only likely to cause more confusion without some explanation to back them up.
Any other opinions?
OneCareBear
Tuesday, September 4, 2007 12:28 AMModerator -
Personally, I could do without the Monthly Report and would be happy with a status of green and the ability to view details through some other means - perhaps the Monthly Report, but called something else and simpler than the Support Log report.
-steve
Tuesday, September 4, 2007 1:25 AMModerator -
Here is my suggestion for monthly report functionality:
For example, I am interested in what Onecare detects as suspicious software. but you should be able to click on the summary number and drill down to the specific details that underlie the calculation, along with explanations. Users should NOT be expected to manually generate a log report, look thru it, and guess what info there might relate to the reported summary. The log report you recommend is overwhelming with details irrelevant to understanding the reported numbers. And the word "suspicious" appears nowhere in the logs.
I'm a professional programmer and here are my guiding principles for any report:
-
easy to use
-
easy to understand
-
addresses relevant, real-world uses
Here are the features I would include in any report:
-
a summary of the targeted audience
-
goals of the report, how it is intended to be used
-
a description of where the data comes from
-
how it is transformed to derive the reported figures
-
drill-down to supporting details to allow manual verification or deeper understanding
The current version does not appear to comply with the principles, and it lacks most of these features.
Can you please forward my suggestions to the oneCare team?
thank you
Sunday, April 6, 2008 2:53 PM -
-
David,
Your guiding principals make good sense, but in this case, the features you listed are mostly what OneCare tries to avoid. Though an analysis of the targeted audience, report goals and usage should guide the choices of data to be displayed, your first four items make the most sense in the scope of a technical audience. In the case of OneCare, since the audience is assumed to be non-technical users this type of formalized reporting is simply too complex.
I do agree that the analysis that was done and the resulting report don't seem to effectively perform the function that we're guessing was their intent. Rather than simplify the information found in the existing technical report it seems to mostly modify and obscure these results even more, which I believe is why Steve indicated he'd rather not see it at all. I'm more ambivalent, but I don't see much advantage to the current version, though it was 'improved' since the time of this thread during the 2.0 beta last summer.
You'll note that I also mentioned the option of drilling down (linking) to more technical information to help educate those who wish to be more informed. This was something I expected to see much more of in OneCare as a method of reducing the FUD and mythology that has developed in relation to malware/anti-malware and the programming field in general. I agree that any improvement in access to real valid information including things like explanations of the various threats and how they can be mitigated might be potentially helpful.
I do find it interesting though that we've seen little discussion of the Monthly Report here since the release of 2.0 last November. This and an apparent reduction in the number of "is it doing anything?" posts might indicate that the reporting is performing as desired, or maybe that users are simply ignoring it completely.
OneCareBear
Monday, April 7, 2008 5:52 AMModerator -
I see your point about this being targeted at a non-technical audience. I am not sure how such a person feels or what they want so I will assume Microsoft has interviewed folks and knows. I agree that either we get links to more specifics or thy should remove it. To say X suspicious things were found without a link to supporting data or more explanation is to alarm the user. I don't know whether my machine has been compromised or not.
Perhaps the reason we see less of the "is it doing anything?" posts is because it's been on our machines for some time now and we are not being overrun by viruses. So common sense says it is probably working. THough the cause could just as well be user awareness and good browsing hygiene.
Monday, April 7, 2008 11:46 AM -
I just wanted to point out that I tend to agree with you, David. As far as I'm concerned the monthly report should be a option within the logging tab under change settings and it should provide much more detail. As it stands now, it looks pretty, but it doesn't really give me any decent information and does have a tendency to alarm people with the unsupported threats found entry count.
-steve
Tuesday, April 8, 2008 3:19 PMModerator -
You could always uncheck the box in settings and logging and never see a monthly report again
Tuesday, April 8, 2008 4:29 PM