If you’re having trouble posting something because pastebin says it looks like spam, post a sample in a comment below and I’ll see what I can do to improve it!

Last night I built in some spam filtering which has caught hundreds of posts since going live. I also added a “report spam” link which has flagged over 500 posts in past 20 hours. By iteratively tweaking the spam filter to identify the legimately flagged posts, I’ve been able to quickly delete a lot of older spam posts.

Hopefully this will make pastebin look like a well tended garden rather than a run-down wasteland! Comments welcome…

It’s nice to see that people still want to use it, so I’m going to ride this wave of enthusiasm and improve it further over the coming weeks.

Your feedback, as ever, is welcome!

• MySQL storage replaced with file-based storage, making it much faster
• Revamped the colour scheme, which has been pretty much the same for 5 years
• Added a ‘delete post’ feature
• Switched to Affero GPL licence

If you’ve drifted away from pastebin due it’s lethargic speed, now’s the time to come back! Give it a whirl and if you have any feedback, leave a comment on this post.

Here’s some more detail on the changes…

File based storage

Pastebin used MySQL for storage since it was first launched in 2002. It has steadily grown in popularity, but that popularity began to take its toll on performance in the past 12 months.

Pastebin started out just keeping the last 1000 posts, which kept things zippy. Then I added custom domains, which increased the number of posts being retained, but what really hurt it was adding a common request – permanent posts, which meant that over time, the database grew inexorably larger.

In January I began to wonder if I needed a relational database at all. After all, pastebin is really just a single table application, and there are only two main operations:

• Fetch post x
• Get last 10 posts on domain foo

So I refactored the code to allow the storage mechanism to be changed. The new file based mechanism assigns a random identifier to a new post, e.g. abcdefgh and stores it in a structured directory:

posts/<d|m|f>/ab/cd/ef/abcdefgh

The top level directory ‘d’, ‘m’, or ‘f’ is chosen based on the desired lifetime of the post (1 day, 1 month or forever). Garbage collection of the 1 day posts in the ‘d’ directory can thus be carried out by performing a find for files older than a day with something like this running from cron every day:

find /path/to/pastebin/posts/d -mtime +1 -exec rm \{\} \;

To maintain the MRU lists of recent posts, the code maintains a serialized array for each domain. Whenever a post is made, this serialized file is locked, updated and unlocked. This is the only time the code can find itself competing for a shared resource, and even then its on a per-domain basis, rather than for the entire application as with the mysql storage.

As I write, this mechanism has been running for a few hours on the live site, and performance is much improved. At peak times it could take 15-20 seconds to make a post, it’s now much, much zippier!

Revamped Colour Scheme

I thought the old CSS was looking a little tired so I’ve freshened it up a little. I want to avoid adding graphics to the design and just use pure HTML and CSS if possible, which keeps things speedy too.

Comments on it are welcome, it’s likely I’ll tinker with it some more…

Delete Post

This is quite neat I think – if you choose to hit the “remember me” button, you’ll be assigned a random token which is used to mark your posts. This token is stored in a cookie. When you later view a post, if your cookie token and the post token match, you’ll be offered the opportunity of deleting the post.

I like this as you don’t have to go entering a password or setting up an account – it just works.

As always, if you’ve made a post you want removing and this feature doesn’t do it for you, just ask and I’ll take care of it

Changed to Affero GPL

The last few releases of pastebin used the GPL licence. Trouble is, while the GPL guarantees access to the source if you receive a binary copy of the software, with a website that doesn’t happen. The Affero GPL is a modified version of the GPL which contains an extra clause guaranteeing your access to the source when you interact with the software over a network.

So if you use pastebin in your own site, or adapt it further, you must continue to offer that source to your users. Lovely

What’s next?

Well, now that pastebin is actually usable again, I’m on a roll. The code has partially complete support for translation, and I’ve an army of volunteers ready to translate, so that’s the next goal…

Over the next few days I’m rewriting the internals to be much faster, and hopefully release it over the weekend.

Wish me luck!

Thanks to the (many!) folks who alerted me!

Edit: OK, I’ve restored a backup from a few weeks ago, which means most recent posts are gone. Many apologies for the inconvenience caused, still looking for the root cause (all the pastebin tables were just *gone* ).

I’ve made a number of fixes to address the problem, if you notice further issues, do let me know with a comment below.

(EDIT: I’ve had offers for Albanian, Arabic, Simplified Chinese, Danish, Dutch, French, German, Greek, Italian, Japanese, Polish, Portuguese, Russian, Spanish, Swedish,Turkish and Urdu – thanks folks!)

• 33% expire in a day
• 64% expire in a month
• 3% never expire

In a few days I’ll take a look at some of the “permanent” posts to see what other features might be useful…in meantime, feel free to make suggestions by commenting below!

I’ve rectified this in v0.50. There are three options to suit various users

• expiry in a day – great for IRC or IM, where the post will be read pretty much immediately and then fall out of use
• expire in a month – great for exploring code ideas or problems via email
• never expire – the post lives forever. This is really for retaining code snippets you want to share with others.

I’m going to add more features that will make pastebin more useful for long term storage of code snippets, watch this space!

Download GPL’d source code, or just play with it at pastebin.com