Since FriendFeed was bought by Fackbook, it's doomed for itself and users. Yes, they still have a team to maintain and fix issues, but it has become a zombie I would say. I stayed until two months ago, I removed all services I had added and removed it from yjl.im. As I see it as a zombie, it continued grabbing my stuff from a few sources for a few weeks. I didn't report because I didn't care.
Few days ago, I decided to remove some entries. I have wanted to do that for long time after I saw many links from FriendFeed (to this blog) reported in Webmaster Tools. Surely, I would have those link, I added my blog to FriendFeed. It's same reason why often don't link to my blog post in screenshots I upload on Flickr. I feel I was spamming myself. Time to fix it.
I am not trying to remove all entries but only those without being commented on or liked, I also don't remove the FriendFeed entries. For example, you write an entry or upload files or images directly on FriendFeed. I want to keep everything original intact. Blog entries, YouTube favorites, Last.fm favorites, etc., they will be still intact in source websites as they were after I remove those entries from my FriendFeed account. But likes and comments on FriendFeed are original stuff, so I keep entries which have those.
Unfortunately, there is an API rate limit for deletion (or write permission). Of course I use API, you don't expect me to delete 9,430 entries (10,091 entries in total in my account) by mouse clicks, do you? I don't know exact rate, it seems to be 100 requests per a few hours or per a day, I am not sure. Conservatively speaking, it's a 100-day job. I wrote an email to API team for detail about rate limit but I haven't got an response.
Since this is a one-time script, I just post the code here.
#!/usr/bin/env python
import datetime
import getpass
import re
import shelve
import sys
import time
from urllib2 import HTTPError
from friendfeed import FriendFeed
NUM = 100
DELETION_INTERVAL = 30
RE_TITLE = re.compile('(.*) - <a rel.*')
def print_eta(n, extra=0, est_only=False):
return
eta = 3600 * n / 100 + extra
est = datetime.datetime.now() + datetime.timedelta(seconds=eta)
if est_only:
print '[%s]' % est
else:
print 'Estimated time to complete: %d seconds, at %s' % (eta, est)
def main():
ff = FriendFeed()
nickname = raw_input('Your FriendFeed Nickname: ')
data = shelve.open('%s.data' % nickname)
if 'start' not in data:
start = 0
else:
start = data['start']
if start == -1:
# Finish retrieving entries
entries = data['entries']
marked = len([True for v in entries.values() if v[0]])
total = len(entries)
del_queue = [entry for entry, value in entries.items() if value[0] and not value[1]]
print '%d out of %d entries marked for deletion.' % (marked, total)
print '%d deleted, %d left to delete.' % (marked - len(del_queue), len(del_queue))
print
if not del_queue:
return
print 'You can find your Remote Key at http://friendfeed.com/remotekey'
print
remote_key = getpass.getpass('Please enter your remote key [no echo]: ')
ff = FriendFeed(nickname, remote_key)
print
print_eta(len(del_queue), extra=5)
print 'Starting deletion (every %d seconds a request) in 5 seconds...' % DELETION_INTERVAL
print
time.sleep(5)
del_count = 0
try:
while del_count < len(del_queue):
e_id = del_queue[del_count]
try:
result = ff._fetch('/api/entry/delete', {'entry': e_id})
except HTTPError, e:
data['entries'] = entries
data.sync()
if e.code == 403 and 'limit-exceeded' in e.read():
print
print 'Failed to delete [%s], reached the rate limit.' % e_id
print_eta(len(del_queue) - del_count, extra=10*60)
print 'Sleeping for 10 minutes...'
time.sleep(10 * 60)
print
continue
raise e
if result['success']:
entries[e_id] = (True, True)
else:
print
print 'Failed to delete [%s]: ' % e_id, result
print 'Continue, anyway.'
sys.stdout.write('#')
sys.stdout.flush()
del_count += 1
if del_count % 50 == 0 or del_count == len(del_queue):
sys.stdout.write(' %d \n' % del_count)
print_eta(len(del_queue) - del_count, extra=10*60)
data['entries'] = entries
data.sync()
time.sleep(DELETION_INTERVAL)
except Exception, e:
data['entries'] = entries
data.sync()
raise e
print 'Done.'
else:
entries = data.get('entries', {});
# Retrieving entries
while True:
feed = ff.fetch_user_feed(nickname, start=start, num=NUM, maxcomments=1, maxlikes=1, hidden=1)
ids = [entry['id'] for entry in feed['entries']]
for e_id in ids:
if e_id not in entries:
break
else:
# All already in entries
print 'Retrieval is done.'
break
for entry in feed['entries']:
if entry['id'] in entries:
continue
if entry['service']['id'] == 'internal':
# Don't delete FriendFeed stuff
entries[entry['id']] = (False, False)
#elif 'comments' not in entry and 'likes' not in entry:
elif len(entry['comments']) + len(entry['likes']) == 0:
entries[entry['id']] = (True, False)
else:
entries[entry['id']] = (False, False)
print 'start=%d, entries=%d' % (start, len(entries))
start += NUM
data['entries'] = entries
data['start'] = start
data.sync()
time.sleep(5)
data['start'] = -1
data.sync()
marked = len([True for v in entries.values() if v[0]])
total = len(entries)
print '%d out of %d entries marked for deletion.' % (marked, total)
data.close()
if __name__ == '__main__':
main()
(I silence print_eta(). Because I don't have exact rate limit information, therefore I could not give an ETA.)
It's a two-stage design. First run is to collect entries and to mark entries should be deleted, you will only be asked for FriendFeed nickname. The collected data will be stored in nickname.data file.
$ ./remove_lonely_entries.py Your FriendFeed Username: livibetter start=0, entries=100 start=100, entries=200 start=200, entries=300 start=300, entries=400 [...] start=9600, entries=9691 start=9700, entries=9791 start=9800, entries=9891 start=9900, entries=9991 start=10000, entries=10091 Retrieval is done. 9430 out of 10091 entries marked for deletion.
The second stage is to delete entries, you will also be asked for remote key. I still use API v1, API v2 uses OAuth and I am not sure if it supports three-legged OAuth. No need to trouble myself to that. Once you enter the key, the deletion will be started in five seconds. It will send a deletion request every 30 seconds. If it gets a rate limit exceeded response, it sleeps for 10 minutes, then try again.
It should be safe to interrupt this script anytime you want, it will pick up where you force it to leave. There is no option to tell this script which stage to perform, it knows. Simply run the script without options and follow the instruction, it will be fine.
Meh, I still have 8,887 entries to delete!