sync status report

[meeotch]

A great feature would be an additional "status" window that's echoed to the logfile after each sync.  Something that gives a summary so you can quickly tell if everything went o.k., or if you need to dig through the logs to check on possible problems.  E.g.:

   Added to Thunderbird  3
   Deleted from Thunderbird  0
   Updated in Thunderbird  4
   Added to GContacts 0
   Deleted from GContacts 2
   Updated in GContacts 3
   Conflicts 2

Or even just "deleted" and "conflicts"...  So anything that might have resulted in data loss.  (Conflicts being any contacts that were updated in both locations since the last sync.)  

Currently, even the non-verbose version of the log file is too verbose to want to wade through searching for problems every time you sync - particularly if you have a lot of books/groups.  Also, I believe that deleted contacts aren't specifically flagged as such in the non-verbose log - so it's pretty tough to find them, even if you are wading through your logs.

After your initial sync, I feel like it's generally the case that you have a rough idea what's changed since the last time you synced.  So you can quickly look at a short summary like the above and say, "Nothing deleted, no conflicts - good."  or "Hey!  I don't remember deleting anything!"  (Obviously, this would be for manual syncs only.)

mitch

[Josh Geenen]

Hey,

Thanks for the suggestion, I agree that it would be useful so I'm adding it to version 0.3's log.

How does this example from my log look?

Code: [Select]

*****Summary for 'Address Book'*****
Not changed: 76
Conflicted:  0 (changed in Thunderbird and Google)
Changes to Thunderbird:
 - Added:   0
 - Updated: 1
 - Removed: 3
 - Ignored: 0 (due to read-only mode)
Changes to Google contacts (remote):
 - Added:   0
 - Updated: 2
 - Removed: 0
 - Ignored: 0 (due to write-only mode)
*****End of summary*****
Version 0.3 makes it significantly easier to synchronize multiple address books with one or more Google accounts.  Would you like an option to display the summary for an address book right after it is synchronized manually or all of those summaries at one time (might be too large to view), or is that just being in the log enough?

I'm also going to clean up non-verbose logging.  There were a few cases where nothing was logged.

Thanks,

Josh

[meeotch]

Holy crap - that's got to be the quickest turn-around ever.  You're the man.  (Go non-corporate software!)

Looks good to me - I might be missing something, but off the top of my head that seems to fit the bill.  I'd personally vote for an (optional) pop-up at the end of every manual sync, in addition to the log.  Just a single total of all address book results, to keep it short.  (Less useful for people who sync many books in opposing directions, but I still feel like it's good enough for a quick "all is well".)

I suppose this will be slightly handicapped by the thunderbird sent-email-counting-falsely-changes-contact-mod-times-forcing-unnecessary-sync problem mentioned in the FAQ.  But I don't know that I've actually observed that behavior, even when I tried to force it one time.

[Josh Geenen]

Ok, here's a sample popup I got after synchronizing my accounts.  The only difficulty now is in aligning the numbers like they are below.  They look fine in monospace (every character has the same width) but it looks pretty bad in a popup.

In the log there are a few entries like this:

Code: [Select]

*****Summary for 'Personal Address Book'*****
 - Not changed: 17
 - Conflicted:  0    (changed in Thunderbird and Google)
Changes to Thunderbird (local):
 - Added:       0
 - Updated:     0
 - Removed:     0
 - Ignored:     0    (due to read-only mode)
Changes to Google contacts (remote):
 - Added:       0
 - Updated:     0
 - Removed:     0
 - Ignored:     0    (due to write-only mode)
*****End of summary*****

Followed by an overall summary:

Code: [Select]

*****Synchronization Summary*****
 - Not changed: 166
 - Conflicted:  0 (changed in Thunderbird and Google)
Changes to Thunderbird (local):
 - Added:       1
 - Updated:     3
 - Removed:     3
 - Ignored:     0 (due to read-only mode)
Changes to Google contacts (remote):
 - Added:       1
 - Updated:     1
 - Removed:     0
 - Ignored:     0 (due to write-only mode)
*****End of summary*****

The popup can't have nice aligned numbers since it doesn't use a monospace font (all characters are the same width) so it looks something like this:

*****Synchronization Summary*****
 - Not changed: 166
 - Conflicted: 0   (changed in Thunderbird and Google)
Changes to Thunderbird (local):
 - Added: 1
 - Updated: 3
 - Removed: 3
 - Ignored: 0   (due to read-only mode)
Changes to Google contacts (remote):
 - Added: 1
 - Updated: 1
 - Removed: 0
 - Ignored: 0   (due to write-only mode)
*****End of summary*****

How does this look?  I'll have a checkbox in the Preferences dialog to let you choose whether you want this summary in a popup after every (or manual only?) synchronization.  It will always be logged.

I suppose this will be slightly handicapped by the thunderbird sent-email-counting-falsely-changes-contact-mod-times-forcing-unnecessary-sync problem mentioned in the FAQ.  But I don't know that I've actually observed that behavior, even when I tried to force it one time.

Thanks for reading the FAQ, I always wonder if anyone reads it .  It has been a while since I looked at the code responsible for the receive count on contacts so it might not still do that.

[meeotch]

Nice one, man. 

or manual only?

Probably.  Folks that keep auto sync on (not me) would probably get irritated otherwise.

The popup can't have nice aligned numbers since it doesn't use a monospace font

Possible workaround would be to flip the format, so the numbers come before the labels.  But honestly, it doesn't seem that hard to read, since the labels are short.

It has been a while since I looked at the code responsible for the receive count on contacts so it might not still do that.

Another possible workaround:  compare the source and destination values before copying.  If they're the same, skip it.  This would also fix the case where you've changed one of the non-synced fields (adding a nickname, for instance), which I think forces an unnecessary update currently.  Or just wait until someone complains.  ;-)

[Josh Geenen]

Probably.  Folks that keep auto sync on (not me) would probably get irritated otherwise.

Good point, I added a preference labeled: "Display a summary after every manual sync" that is on by default for now.

Another possible workaround:  compare the source and destination values before copying.  If they're the same, skip it.  This would also fix the case where you've changed one of the non-synced fields (adding a nickname, for instance), which I think forces an unnecessary update currently.  Or just wait until someone complains.  ;-)

I have been thinking about doing that for a while now, but I would prefer not to do that in 0.3 this close to it being released.  I might in 0.4.

[meeotch]

Thanks!  Will try it out when it's available.

[Josh Geenen]

If you are feeling brave you can try out the pre-release version 0.3.0rc1 which has the sync summary and a lot of new features compared to version 0.2.  If you want to go that route, please read this for more details and make sure you back everything up.  The actual download is in this thread.

Otherwise, if things go well with 0.3.0rc1 I will release 0.3.0 within the next two weeks.

[meeotch]

3.0rc1 seems to work as advertised, on my admittedly small test of two address books.  Will give a shout if anything comes up as I work my way through the rest of my contacts.  (The new status window already alerted me to two contacts that claimed to be conflict-changed on both sides, but which I didn't remember changing at all.)

Unsolicited suggestion:  If you're still in the midst of log-cleaning, I had another thought:  if it's possible, it might be useful to output human-readable dates/times for changes, in addition to / instead of whatever machine format is currently in there.

[Josh Geenen]

3.0rc1 seems to work as advertised, on my admittedly small test of two address books.  Will give a shout if anything comes up as I work my way through the rest of my contacts.  (The new status window already alerted me to two contacts that claimed to be conflict-changed on both sides, but which I didn't remember changing at all.)

Thanks for checking it out!

Unsolicited suggestion:  If you're still in the midst of log-cleaning, I had another thought:  if it's possible, it might be useful to output human-readable dates/times for changes, in addition to / instead of whatever machine format is currently in there.

That is a good point, I forgot that I am not the only one reading the log.  Those numbers are the number of milliseconds or seconds that have passed since midnight on January 1, 1970 UTC.  It's a convenient way for storing and comparing dates and times, but most people can't convert that to human-friendly dates in their heads (I know I can't very quickly ).

If you need to convert one of these timestamps you can do so through the Error Console in Firefox or Thunderbird:

Tools -> Error Console

Type the following, but replace the number below with the number you want to convert.
alert(new Date(1283977771906));
Hit Evaluate or press Enter.

You should get a popup with something like this:
Wed Sep 08 2010 16:29:31 GMT-0400 (EST)

Josh

EDIT: I'll track my progress here and probably forget to update this thread.  It will be in the next release of 0.3.0 (rc2 or 0.3.0).

[meeotch]

That's a cool trick, converting posix time with the error console - didn't know you could do that.  (I'll have to use that next time I'm at a party!  :-)

Thanks for the support, and the plugin - keep fighting the good fight, dude.

[Josh Geenen]

That's a cool trick, converting posix time with the error console - didn't know you could do that.  (I'll have to use that next time I'm at a party!  :-)

I have used it at quite a few parties already. 

FYI I forgot new in that post, so I just updated it.

[Josh Geenen]

Here are some examples of what you might find in a log now as of version 0.3.0 and above:

You can find the latest version (currently 0.3.0) here: https://addons.mozilla.org/en-US/thunderbird/addon/8451/ by Monday.

Code: [Select]

Starting Synchronization for username@gmail.com at: 1294514124768 (Sat Jan 08 2011 14:15:24 GMT-0500 (EST))
...
Last sync was at: 1294513975729 (Sat Jan 08 2011 14:12:55 GMT-0500 (EST))
...
Josh Geenen: http://www.google.com/m8/feeds/contacts/username%40gmail.com/base/133b6a4608bd807d
 * Found a match, last modified:
   - Google:      1293634864809 (Wed Dec 29 2010 10:01:04 GMT-0500 (EST))
   - Thunderbird: 1294426653000 (Fri Jan 07 2011 13:57:33 GMT-0500 (EST))
 * Neither contact has changed
Sorry this took me so long to get to, and thanks for the suggestion!