Mailing List Archive Howto - Revision history

Srevilak: /* public archives */

2013-05-12T23:46:25Z

public archives

← Older revision		Revision as of 23:46, 12 May 2013
Line 20:		Line 20:

	Here's an example of archiving a public mailing list. You'll run		Here's an example of archiving a public mailing list. You'll run
	ob-archive-list with the list's archive URL		ob-archive-list with the list's archive URL. For this example, the archive URL is https://lists.mayfirst.org/pipermail/fawg/.

	<pre>		<pre>

Srevilak: /* non-public archives */

2013-05-12T23:45:30Z

non-public archives

← Older revision		Revision as of 23:45, 12 May 2013
Line 58:		Line 58:
	* Copy mailman's cookie		* Copy mailman's cookie
	* Provide that cookie as a second argument to ob-archive-list		* Provide that cookie as a second argument to ob-archive-list

			I'll used https://lists.mayfirst.org/mailman/private/everyone-submit/ for illustration.

	Here's an example of a mailman cookie. I've blacked out portions of		Here's an example of a mailman cookie. I've blacked out portions of

Srevilak: /* Archiving mailing list archives */

2013-05-12T23:44:29Z

Archiving mailing list archives

← Older revision		Revision as of 23:44, 12 May 2013
Line 15:		Line 15:
	Now, let's get down to work. There are two cases to consider		Now, let's get down to work. There are two cases to consider
	# public archives (those accessible without a password), and		# public archives (those accessible without a password), and
	# non-public archives (which require a username and password to		# non-public archives (which require a username and password to access)
	access)

	== public archives ==		== public archives ==

Srevilak: new section: Mailing list deletion

2013-04-01T00:46:32Z

new section: Mailing list deletion

← Older revision		Revision as of 00:46, 1 April 2013
Line 95:		Line 95:
	Afterwards, look at http://archive.occupyboston.org/mailing-lists/, to		Afterwards, look at http://archive.occupyboston.org/mailing-lists/, to
	make sure the .tgz uploaded correctly		make sure the .tgz uploaded correctly

			== Mailing list deletion ==

			After you've uploaded a mailing list archive, it's worth taking a
			moment to make sure the archive uploaded correctly. For example,
			after archiving LIST, do "ls -lh LIST.tgz". Compare the size of your
			local copy with the one on
			http://archive.occupyboston.org/mailing-lists.

			Once you're sure the archive copy is good, there is one remaining
			step: deleting the mailing list.

			* https://members.mayfirst.org/cp
			* Log in
			* Select the "obmail.org" hosting order
			* Click the "Email List" tab
			* Find the list you want to remove
			* Click "Delete"

			Delete is permanent, so be careful, and make sure you're deleting the
			right list!

Srevilak: initial revision

2013-04-01T00:26:49Z

initial revision

New page

= Archiving mailing list archives =

Download a copy of
http://archive.occupyboston.org/mailing-lists/ob-archive-list.

Create a new directory for your archiving work; if something goes
wrong, this avoids the possibility of having (say) thousands of html
files littering your archive directory.

mkdir work
cp ob-archive-list work
cd work

Now, let's get down to work. There are two cases to consider
# public archives (those accessible without a password), and
# non-public archives (which require a username and password to
access)

== public archives ==

Here's an example of archiving a public mailing list. You'll run
ob-archive-list with the list's archive URL

<pre>
$ sh ob-archive-list https://lists.mayfirst.org/pipermail/fawg/
ob-archive-list: downloading https://lists.mayfirst.org/pipermail/fawg/
2013-03-31 19:54:24 URL:https://lists.mayfirst.org/pipermail/fawg/ [1650/1650] -> "fawg/index.html" [1]
2013-03-31 19:54:24 URL:https://lists.mayfirst.org/pipermail/fawg/search [5925] -> "fawg/search" [1]
2013-03-31 19:54:24 URL:https://lists.mayfirst.org/pipermail/fawg/2013-March/thread.html [10065/10065] -> "fawg/2013-March/thread.html" [1]
2013-03-31 19:54:24 URL:https://lists.mayfirst.org/pipermail/fawg/2013-March/subject.html [7897/7897] -> "fawg/2013-March/subject.html" [1]
2013-03-31 19:54:24 URL:https://lists.mayfirst.org/pipermail/fawg/2013-March/author.html [7899/7899] -> "fawg/2013-March/author.html" [1]
...
2013-03-31 19:54:31 URL:http://lists.mayfirst.org/pipermail/fawg/attachments/20130227/d5702cea/attachment.xls [12800/12800] -> "fawg/attachments/20130227/d5702cea/attachment.xls" [1]
2013-03-31 19:54:31 URL:http://lists.mayfirst.org/pipermail/fawg/attachments/20130227/d5702cea/attachment.pgp [198/198] -> "fawg/attachments/20130227/d5702cea/attachment.pgp" [1]
FINISHED --2013-03-31 19:54:31--
Total wall clock time: 6.9s
Downloaded: 187 files, 901K in 1.0s (929 KB/s)
ob-archive-list: downloaded 187 file(s) from https://lists.mayfirst.org/pipermail/fawg/
ob-archive-list: creating fawg.tgz
ob-archive-list: uploading fawg.tgz => obarchive@archive.occupyboston.org:archive.occupyboston.org/web/mailing-lists/
X11 forwarding request failed on channel 0
fawg.tgz
233700 100% 47.91MB/s 0:00:00 (xfer#1, to-check=0/1)

sent 233800 bytes received 31 bytes 93532.40 bytes/sec
total size is 233700 speedup is 1.00
ob-archive-list: DONE
</pre>

Afterwards, you should see a .tgz of the mailing list archives in
http://archive.occupyboston.org/mailing-lists/.

== non-public archives ==

Non public archives require a little more work. You'll need to

* View the archive URL in your web browser (which requires authentication)
* Copy mailman's cookie
* Provide that cookie as a second argument to ob-archive-list

Here's an example of a mailman cookie. I've blacked out portions of
the cookie name and Content (aka, the cookie value), but those are the
pieces you need. Firefox allows you to select and copy the values,
and that's what you'll need to do.

[[File:Mailman-cookie.png]]

Here's an example of archiving a non-public list. Note that the
second argument to ob-archive-list is the cookie Name=Content.

<pre>
$ sh ob-archive-list https://lists.mayfirst.org/mailman/private/everyone-submit/ everyone-submit+user+jsmith--at--example.org=28020000006948c558517328000000363464363831643863643364000000000000000000061653964366165623239316462663439333630
ob-archive-list: downloading https://lists.mayfirst.org/mailman/private/everyone-submit/
2013-03-31 20:16:38 URL:https://lists.mayfirst.org/mailman/private/everyone-submit/ [4289] -> "everyone-submit/index.html" [1]
2013-03-31 20:16:38 URL:https://lists.mayfirst.org/mailman/private/everyone-submit/search [5964] -> "everyone-submit/search" [1]
...
2013-03-31 20:16:57 URL:https://lists.mayfirst.org/mailman/private/everyone-submit/attachments/20120827/d74bc7d2/attachment.htm [6451] -> "everyone-submit/attachments/20120827/d74bc7d2/attachment.htm" [1]
2013-03-31 20:16:57 URL:https://lists.mayfirst.org/mailman/private/everyone-submit/attachments/20120830/c68b69b6/attachment.htm [44740] -> "everyone-submit/attachments/20120830/c68b69b6/attachment.htm" [1]
FINISHED --2013-03-31 20:16:57--
Total wall clock time: 19s
Downloaded: 92 files, 1.9M in 3.3s (579 KB/s)
ob-archive-list: downloaded 92 file(s) from https://lists.mayfirst.org/mailman/private/everyone-submit/
ob-archive-list: creating everyone-submit.tgz
ob-archive-list: uploading everyone-submit.tgz => obarchive@archive.occupyboston.org:archive.occupyboston.org/web/mailing-lists/
X11 forwarding request failed on channel 0
everyone-submit.tgz
1483299 100% 43.23MB/s 0:00:00 (xfer#1, to-check=0/1)

sent 1290846 bytes received 7351 bytes 288488.22 bytes/sec
total size is 1483299 speedup is 1.14
ob-archive-list: DONE
</pre>

Afterwards, look at http://archive.occupyboston.org/mailing-lists/, to
make sure the .tgz uploaded correctly