Topic: exporting files using API

Here is another vote for a way to export multiple/all files at once. There are certainly some legitimate bandwidth concerns if everyone were to download all files at a high frequency. If this is the main concern maybe the solution would be to put some limits on such a service, (and if needed offer additional usage for a modest fee.) Here is a specific proposal to start with (feedback/improvements welcome):

1. Allow all users to download up to a fixed number of files (~5 or 10?) or a max total size (to allow lots of small files) simultaneously. This would simplify downloading multiple project files for legitimate work
2. Provide the account owner a limited number of opportunities (1/week?) to dump the entire file collection for backup purposes.
3. Allow a limited number of downloads when a project is archived.

Thoughts?

Bryan

Topic: exporting files using API

I’m very happy to read 37signals response on this thread – I think it was a long time coming.

While I share some of the frustrations of previous posters, I am sympathetic to 37signals too because of how they host our files and their price / cost model. 37signals doesn’t store our files on their own servers, they use Amazon’s Simple Storage Service [S3]. While we pay a fixed monthly fee to 37signals, they are charged a variable fee by Amazon based on the volume of data stored plus GBs uploaded or downloaded. Controlling downloads to single files via manual user requests is 37signals’ way of controlling their own costs and the price they charge us.

Amazon’s charges appear to be very reasonable at $0.15 per GB stored and $~0.10 per GB transferred BUT please imagine the impact we could have on 37signals’ costs if we could bulk download ALL our files at will! As David says above, a Basecamp Max account can store 50-GB of data. If a full backup was taken every night we would be downloading 1.5-TB per month which would cost $150, i.e. $1 more that the Max account fee of $149 per month. That’s not good business for 37signals :-).

Of course that’s worse case and you could reduce the amount of data by reducing the frequency of backup or working out what had changed each day, week or month but it’s still an unpredictable cost to 37signals who would be forced to increase their charges. But then perhaps it should be a cost option for us to choose if we’re prepared to pay.

And finally, while we’re dependent on 37signals for access to the data, we’re actually dependent on Amazon for the hosting and security of that data: that reassures me.

In short: I think we’re asking for the wrong feature but I can’t quite formulate what the right one should be (yet!).

Regards,

Ian.

Topic: exporting files using API

Hi guys,

We’re definitely interested in expanding our backup facilities. We’ve talked about a few ways of doing this including allowing you to download a mega zip file with all files from all projects. The problem, of course, is that this file would be humongous. Up to 50GB for the Max plan. So it’s not an entirely trivial thing to do.

Another option as mentioned is to list all the attachments as part of the export, but would require programming talent to convert into an actual backup. We’d like to move away from that and have backups be something everyone can easily use without being a programmer.

In any case, we’re looking into this, but I can’t (as with any feature) promise the whens and the hows.

I can say, though, that we’ve been running Basecamp for more than 4 years and been in business for almost a decade (so we’re not some fly-by-night startup that’s going under tomorrow). Your files are very safe with us. That doesn’t mean better backups is not a reasonable request, though. I can totally understand why people want to be able to get their data out and we want to help them to do so.

Topic: exporting files using API

We ALL have waited and waited on this issue 37 Signals.

Clearly 37 Signals does not understand the business world on this issue. We all have risks that must be managed including data recovery. We have to be able to restore all data should a data service like 37 signals go out of business next week.

THIS WILL HAPPEN and you will ALL lose ALL your data.

We have been working on our own version of this very good and simple version of this program. It WILL allow people to backp their data by simply adding a table of all attachments that is easily read. Then by simply pressing one button in Access it will download all data to wherever you need it backed up.

This has to be the most ridiculous cross armed uhn-uhn approach to what has been aked for for years now by its customer base by any business that wants happy customers.

SHAME ON YOU 37 SIGNALS. Such a simple thing.

Topic: exporting files using API

I need this functionality as well. Kind of surprised that no one from basecamp has responded to this thread. If I am going to trust basecamp with MY data then I would like to know that I do have open access to MY data at any time, how hard would it be to dump a list of files via the api?

Topic: exporting files using API

You all are not alone… While it is not much help, here is the message I got from Basecamp support:

My message:

On Mon, Aug 4, 2008 at 10:20 AM, Josh Miller <JoshM@haanmc.com> wrote:
> Can you recommend a good third-party website that makes a complete
> downloadable backup per project? We use Basecamp sooo much that we
> need to start archiving the files and messages in our internal system.
> If you could make a recommendation, that would be great. I understand
> that you don’t neccessarily have any plans for this kind of export.
> And, yes, I also understand that I can go here:
>
> http://www.basecamphq.com/help/general/#exporting_data
>
> but I do wonder if you have come across a reputible company that can
> help me further access my files easily?
>
> Josh
>

Their response:

Hi Josh,

Sorry, but we don’t know of any programs that offer this service.

For a more timely response, please use our support form to submit future help requests in lieu of sending a direct email:
http://www.basecamphq.com/help/support

- Sarah

Topic: exporting files using API

I think this may be the most important matter 37signals should work in… because it is always a concern for everybody to make COMPLETE backups of the information… the XML is Ok, but is not enough and requires an additional step for us to take the API and write some lines to read the posts and additional basecamp information of our projects.

Topic: exporting files using API

I’d be really interested if this is possible also.

Topic: Get listing of files / messages / projects?

Ramya,

There’s a PHP example and a PHP wrapper, linked to on the Code Examples page of the unofficial API wiki.

[advertisement removed – admin]

Topic: Get listing of files / messages / projects?

hi,
i too need help in this.. i’ll be using PHP for this kind. can anybody pls help me to do this?

Topic: exporting files using API

This is an incredibly important topic, and would definitely be interested in the answer. I believe that this issue could be a dealbreaker with this software.

Topic: exporting files using API

Has any one figures this out yet man if i could export messages and download zips of file areas that would be sweet! like some sort of system like easynews (if youve ever used it before) wherein you could select sets of stuff to zip and download ;)

(Keeps singing the beachboys :)

Topic: exporting files using API

Just reading thru the Highrise API and came across the following under the Get Notes documentation. Now wouldnt it be nice if we could do this (Sing in a beach boys melody)

Topic: File icons

Hi,

Is it possible for the Basecamp crew to add icons for file types not currently recognized by Basecamp?

In example, we use the open standard ODF in our company, and would very much like to have icons in the «Files» section for the ODF document types, that is:

• ODT (Open Document Text)
• ODS (Open Document Spreadsheet)
• ODP (Open Document Presentation)
• ODG (Open Document Graphics / Drawing)

Best regards,

Martin Bekkelund, Norway

Topic: FTP Storage - No discussion allowed?

Since ftp is still working (we’re using it – and I wouldn’t have known it was being grandfathered unless I’d stumbled on this thread), 37 Signals could have quietly contacted Sebastien and activated ftp for his old account. The trouble of one more ftp-using account would seem to pale in comparison to the fallout of alienating a formerly-passionate Basecamp advocate. In my opinion this is bad business. Thankfully it isn’t my business.

Topic: FTP Storage - No discussion allowed?

We are new to using this product. We are amazed at the functionality. We are three weeks into testing and have hit a snag with one of our biggest clients. I am not sure this is the appropriate place to ask this so please redirect me if not.

Our client’s firewall blocks the S3 service. It is a public school and they do not want students to access that service from school. Part of the problem is the authentication does not go down to the file level as I understand it (my understanding may be flawed)

Are there any alternatives to using S3? Or can the authentication be changed? Help me ask the right question here, please.

Topic: exporting files using API

This was my thought as well but I’ve also been unable how to work out how to get a list of files so I can get my techies to build a routine based on this. Anyone from Basecamp willing to help out ?

I don’t see what Basecamp would have to lose by telling us how to export files to our own servers. As a business, we can not continue to take the risk of not having a backup of files and some of our clients demand contractually we do this. Our newish business already has about 50 projects running and it is unsustainable to have to manually download file by file. In past forums Basecamp has argued that as we have uploaded files to Basecamp we must already have a copy. They seem to forget this is collaborative software – we have several hundred users who can upload files and the uploaded files are dispersed across hundreds of servers all around the world. A logistical and version control nightmare.

It would be helpful to at least know if file export functionality is even on the drawing board for Basecamp. If so, what is the expected delivery time frame.

Topic: Export/download all files

Yes I have been trying to create archive of my projects for contractual reasons. I originally thought I only needed to dump the XML. The problem as glawrie states is that there is no relation between the messages in the XML dump and the attachments. It tells you there is an attachment associated w/ the message but nothing more. :(

Topic: exporting files using API

Well, you definitely can’t access files through the API last I checked, so I’m not sure what they’re referring to.

Also, unless they don’t want you to export your data (very possible), I don’t see why they wouldn’t tell you how to do it.

Topic: exporting files using API

The helpful folk at 37signals tell me that there are several ways of exporting all the files of all your projects in a batch operation. However, they won’t tell me what they are:) They suggest a forum user might be able to help.

We need to perform a backup our projects. In particular, we need to back up the files attached to each project.

Can anyone tell me how they have achieved this?

Topic: FTP Storage - No discussion allowed?

Hi Melvin,

Got your point. I guess I am frustrated that after having had to push for using BC in my company and with our customers, I now look like a fool because one of the critical issues I had to defend and said was not one, is indeed an issue that is not solvable. After having spent so much energy convincing users, detractors, legal departments, internal policy makers, etc. this was just the straw that broke the camel’s back.

As for my qualification of Jason’s “copy/paste” answer, I was just tired of looking my words in English – in French, “langue de bois” gets translated into “set language” by Babel, not sure it is what I wanted to convey, but more politically correct than referring to some old Kremlin demons. Thanks for reminding me that I went too far on the “ready-made speech”.

Apologies for that.

It is one thing to be convinced there is always a solution, it is another to cope with the fact that there isn’t.

And yes for now, I will have to vote with my feet, disappointed, disillusioned and dissatisfied.

I can only hope that BC becomes so much more successful that this storage question gets revisited and the conclusions are changed, as I have strong doubts that regulations/laws will become more flexible on this topic.

Topic: FTP Storage - No discussion allowed?

I can understand the reasons (from 37signals point of view) not continue the feature in the future.

BUT – the month i signed in (and the month after that) i had the chance to use my own ftp-server (and it was one reason i choosed an promoted basecamp in this times). I never reached any message, that this feature will be phased out. From my point of view it depends on me when I start to use a feature which belongs to the package im paying for. I payed the same sum of money every month and i think i should get the same features also. Now I’ running out of diskspace – im “forced” to upgrade my account. Thats not OK.

I am really disappointed.

Topic: FTP Storage - No discussion allowed?

yeks! i stopped by home for a minute so i thought i’d check email & google news reader and noticed a heated discussion… and WOW, 37s is getting slammed here.

Calling their response a “communist party leader speech” is completely uncalled for. Jason’s response was not like a “communist party leader speech.”

It was clear to me from Jason’s reply that:

- the reason “discussion of adding this service back as a feature request is not OK” is because there are other threads talking about it.

- “what happens to users who cannot use the Amazon storage system?” is that they’ve “made the decision and phased out FTP” so find a different solution or live with their solution or repost in existing threads. (i can understand not finding other threads… mostly because we dont have time to keep digging.)

- “why you don’t just go other way and simply charge these troublesome users who can’t configure an FTP server more to use their own storage…” is because they have made a business decision on what is best course of action for them is to “phased out FTP.” Respect the decision or move on… like u’ve done with other products/services u don’t like.

Personally I don’t have a big problem with the BaseCamp storage method but I can understand that certain companies… especially financial companies, have lots of requirements.

I also understand it must be really frustrating finding a product as good as BaseCamp that you can’t use… so i feel you on that. I have similar feelings about HighRise since they don’t have certain things I need. You just have to voice you problems and either live with it or move on.

Calling their response a “communist party leader speech” is uncalled for.

Topic: FTP Storage - No discussion allowed?

Hi BryanJ,

I think you have a point. It is definitely within BC’s right to alter the service they deliver. I think the question here is more related to client satisfaction rather than a legal / business aspect.

• Is this a change that should be revisited as if affects client satisfaction?
• Is there any plan to address the needs / concerns of clients who need to have their own storage rather than the BC solution?
• Has anyone facing this issue found a suitable alternative to BC ?

From what I gather here, this is not the case. And to be honest, whilst the right of BC are not questioned (“fait du prince”), the answer to the question was not up to my expectations. I learned the term boiler plate thanks to Glawrie, what went through my mind before knowing that term was more like “communist party leader speech”. Sounds sensible, just off the mark / not answering the question.

Anyway, it seems this is the end of the discussion here.

Topic: FTP Storage - No discussion allowed?

37signals’ response seems reasonable and fair to me.

They grandfathered in people who were using it and don’t offer it anymore moving forward for those who weren’t.

This changing/grandfathering is common for services. Basecamp is a service, it’s not software. Software is something you download on your computer and it never changes until you decide update it. Basecamp is a service that you subscribe to. It changes often. All changes can be seen as good or bad depending on where you stand, but that’s part of the deal when you use a service. If you want ultimate control you may want to set up your own servers and host your own software.

Take a cell phone or cable service for example. Plans change, options change. If you get 1000 minutes for $50/month, and the cell company decides to discontinue that plan, they let you stay on it but no one else can sign up for it. If a cable company packages HBO/Showtime together, but then decides to split them into separate plans in the future, they let you keep your HBO/Showtime package but no one else can sign up for it. You can’t use something that isn’t offered anymore unless you were using it when it was offered.

In this case, 37signals used to offer remote FTP file storage. They then offered both FTP and Basecamp File Storage. Then 1.5 years later they decided they didn’t like the tricky experience offered by integrating with external FTP servers. They phased out FTP and replaced it with Basecamp File Storage. Those who were using FTP can keep using FTP. Those who weren’t, or those signing up today, don’t have the old FTP option anymore.

Seems reasonable and fair and in line with decisions that have to be made from time to time by service providers.

Just my thoughts.