DropBox Integration in Codea

DropBox Integration in Codea

A few months back, I was struggling with the idea of backing up and restoring code. With Codea 1.5, most of these concerns are now gone (for example, you now have access to the File.IO() space, which was turned off in previous versions), nonetheless there may be reasons to still have DropBox integration.

Word of warning: this was NOT a simple thing to figure out, and it took me a week to both understand what is needed as well as hack together enough code to make it happen.  Additionally, there's still "hinkiness" with the solution, so while it works, it doesn't really feel like it's production-ready.

A version of my code is located at https://gist.github.com/AntonioCiolino/4091848;  it uses some Caching mechanisms that can be removed. I have included the JSON library and a sample main which uses Cider 1. You can strip out the "button code" and walk down the functions manually to register.

What is DropBox?

Living under a rock, eh? DropBox is a cloud-based file service that allows you to upload and sync your data across multiple machines from a single point. DropBox also allows you to determine which directories are able to be synced. Check out http://en.wikipedia.org/wiki/Dropbox_(storage_provider) for more information about this great service.

Create a DropBox application

A DropBox application is a location in your DropBox account where files go to live specific to an application. It's like a separate login and password to a specific place on your DropBox account; apps get a single point of entry and all content lives there.

In order to get this, you have to sign up for a DropBox developer account. Those are (currently) free to any DropBox user, but you will have to create an application and note what it is for.

Note that if you are an old-school DropBox use like I am, your /public directory exists and is wide open for anyone to see. That can be leveraged as a common collaboration spot instead of making an app.

In this case, we will create a new App. Log on to your developer account, select My Apps from the left, and the "Create an app" button. An app dialog should open as below. Select API, and App Folder. (you can use Full DropBox, but I would recommend against it).

Note that TestApp is already taken, you'll have to make your own app name. I went with MyApp00000.
Select Create; this will create the application. The most important piece of information you will see is the App key and App secret. Note that there is a 5 user limit this way, for testing that's fine. You'll have to request production access to remove the limit.

 

Save your Keys!

These are effectively the username and password of the DropBox account (it's a good-enough analogy). This data gets put into the application that is run, which allows the app to log in to DropBox.

That's it for DropBox!

Set up the Codea Project

The trickiness comes in with Codea.

 

Codea has a great function called http.get() and http.request(). These are actually part of Lua but Codea has them compile and that is what we care about. These functions allow Codea to make calls to external sites. In our case, we will leverage DropBox's OAuth endpoints to make and send content.

According to DropBox, you are required to use OAuth 2.0. I tried to do that with much failure. I did find that DropBox still supports it's previous incarnation of OAuth, and that was much easier to use.

Authoring Codea to use DropBox

Overall, DropBox is going to look for the secret and password, no matter where it comes from. In our case, we are going to make a project library that will send that information to DropBox, and once authenticated we will have a window where we can put files. 

Note that this means that any Codea-based app can link to the library and start dropping files right away; this doesn't have to be for only one Codea (or any) project type. There's good and bad in that...

 

The hard part

So, how do you call DropBox and get to files? Is it as simple as call a username/password and log in?

 

No.

 

DropBox (and most other sites) use OAuth, which means that people don't have to put in usernames and passwords, but applications can handle that for you. In DropBox's case, the user registers with DropBox that a specific application is allowed to access their data, and once that is approved, then the OAuth account is approved to access that specific users' information as if the app was that user. Each OAuth key is unique and associated to the approver's account, so when you allow Dropbox to authorize the request (more on that later), DropBox will tie that OAuth key and secret to your account. This is why I would suggest making your key/secret specific to a folder; if somehow it gets out, you are protecting all of your other data.

 

As I mentioned before, we are going to use OAuth 1.0 The main reason is that OAuth 2.0 was not working well (at first), and I wasn't able to figure it out (until I did the SkyDrive integration, many weeks later).

Note that my code uses the saveLocalData/readLocalData as well as the GlobalData features. This is becuase I wanted to allow multiple Codea apps to access the same folder.

The Project

If you don't have a copy of the code from the GIST, it might make sense to grab that now. I'll be walking through most of the commentary in there.

 

Let's start by declaring our class:

 

function DropBox:init(key, secret)

self.appKey = key

self.appSecret = secret

--this is a cache of the querystring with all of the params. This should not ever

--be directly assigned. I did this for speed.

self.params = nil

end

 

Making a helper function. Note that I re-use the toekn and token_secret buckets for request and final authorization; you cans split them out for yourself if it make sit clearer on what is going on.

~~~

--Used to build a base parameter string for any OAuth/Dropbox authenticated call.

-- if these 4 strings don't already exist / are populated, we never really authenticated.

function DropBox:BuildParams()

--if we've already cached our params, just return them.

if self.params ~= nil then return self.params end

--read these from storage.

local accessToken = readLocalData("oauth_token")

local accessSecret= readLocalData("oauth_token_secret")

if (accessToken == nil) then

--read these from storage.

accessToken = readGlobalData("oauth_token")

accessSecret= readGlobalData("oauth_token_secret")

print("-- Using global key from YOURAPP --")

end

assert(accessToken ~= nil,

"You must create a new request and authorize before accessng DropBox information.")

assert(accessSecret ~= nil,

"You must create a new request and authorize before accessng DropBox information.")

assert(appKey ~= nil, "You must assign a DropBox application before accessng DropBox.")

assert(appSecret ~= nil, "You must assign a DropBox application before accessng DropBox.")

self.params =

"?oauth_consumer_key=".. self.appKey ..

"&oauth_token=" .. accessToken ..

"&oauth_version=1.0" ..

"&oauth_signature_method=PLAINTEXT" ..

"&oauth_signature=" .. self.appSecret .."%26" .. accessSecret

return self.params

end

~~~

 

 

Overall, the rules for connecting to DropBox are as follows:

1. Get a request token. this says, "I'm going to request access"
2. Have the USER authorize. This proves that a human is choosing to do the request
3. Get a "real" token and secret (password) to access DropBox. This is what we want to save, so we can access DropBox in the future without re-registering.

 

So I wrote these three functions:

1. DropBox:GetRequestToken()
2. DropBox:DoAuthorize()
3. DropBox:GetAcccessToken()

Get a request token

The first function simply asks DropBox for a token so that we can have a conversation. This "request token" is a login so that DropBox will talk to us to see what we want and if we are going to follow it's rules.

~~~

--Begins the association of an app with DropBox. This gets a request token,

--which is a request to connect to a DropBox app

function DropBox:GetRequestToken()

local params =

"?oauth_consumer_key=".. self.appKey ..

"&oauth_version=1.0" ..

"&oauth_signature_method=PLAINTEXT" ..

"&oauth_signature=" .. self.appSecret .."%26" ..""

http.get("https://api.dropbox.com/1/oauth/request_token"..params,

function(data,status,headers)

--split data into two

token = string.gsub(data, "([^&=]+)=([^&=]*)&?",

function (k,v)

saveLocalData("req_"..k,v)

end)

print("Ready to authorize")

end,

httpError)

end

~~~

 

Authorize the app (HINKY)

Once we have a request token, we can use that to ask DropBox to grant us an access (application) token. The App token is the actual username/password combination that DropBox will associate with your account long term, but first we have to approve it.

 

This is where things get hinky. In order to approve the token, DropBox requires use to go to their website and hit the "OK" button. However, CODEA LOSES CONTROL once a web page is opened. There's no way "back" for Codea to know we approved and there's no way for Safari to send back the request to Codea. This was the step that made me beat my head against a wall for several days.

 

The solution was simple but feels wrong: We open the browser for the user to approve the request, and then when we switch back to Codea, we wait a few seconds so that DropBox has time to store and log that the approval was granted. If you go back too soon, you'll find that the app isn't authorized. One piece of good news is that you get about 2-3 minutes on a Request Token, so you can try to authorize during that time again. In my case, I simply waited 5 seconds before hitting my button.

 

~~~

--Opens browser so user can authorize this request. User must hit OK for the temporary request

--token to be accepted in the next step.

--The user should take a few seconds after the browser approval before returning and hitting the

--GetAccessToken portion of the code, otherwise DropBox will refuse the request. Trying

--to authenticate again after a few seconds does work, though.

function DropBox:DoAuthorize()

local token = readLocalData("req_oauth_token") or ""

local params = "?oauth_token=" .. token

local url = "https://www.dropbox.com/1/oauth/authorize"..params

openURL(url)

print("Ready to GetAccessToken")

end

~~~

 

Get an access token

Once the app it authorized, the app can request DropBox send it the "real" access token. This is how the app will actually log on. After this point, the request token is useless.

~~~

--This is where we get the "real token". This is the token we store in the application so that

--we can call DropBox with our functions. Once through here, we have access to the /sandbox.

function DropBox:GetAccessToken()

local token = readLocalData("req_oauth_token") or ""

local token_secret = readLocalData("req_oauth_token_secret") or ""

local params =

"?oauth_consumer_key=".. self.appKey ..

"&oauth_token=" ..token ..

"&oauth_version=1.0" ..

"&oauth_signature_method=PLAINTEXT" ..

"&oauth_signature=" .. self.appSecret .."%26" .. token_secret

local url = "https://api.dropbox.com/1/oauth/access_token"..params

http.get(url,

function(data,status,headers)

--split data into two

token = string.gsub(data, "([^&=]+)=([^&=]*)&?",

function (k,v)

saveLocalData(k,v)

saveGlobalData(k,v) --so other lua apps can use it!

end)

print("Ready for Query Info")

end

, httpError)

end

~~~

 

Once this is complete, you can test to see if you have access to DropBox. The following function is the simplest test.

~~~

---once we have the proper keys, we can actually read data and get metadata from DropBox.

function DropBox:GetInfo(callback)

local params = self:BuildParams()

local url = "https://api.dropbox.com/1/account/info"..params

print("Get Info")

DropBox:_GetJSONResult(url, callback)

end

~~~

 

 

The rest of these functions are simply to make helpers to the DropBox API. Examples of chaining these together or nesting

~~~

--save a file to DropBox. If the file already exists, we will get an error.

--Filename is rooted to /sandbox; we are not doing any /dropbox (global) saves.

--We didn't call JSON here because we have to pass a bucket of data and we want to stay

--as clean as possible here.

function DropBox:Write(filename, data, callback)

local cb = callback or nil

params = self:BuildParams()

local tbl = {["method"]="POST", ["data"]=data}

--save the contents

http.get("https://api-content.dropbox.com/1/files_put/sandbox/".. filename .. params,

function(data,status,headers)

result = {}

result= JSON:decode(data)

if cb ~= nil then cb(result) end

end,

httpError, tbl)

end

--Reads a file, if a callback routine is supplied, calls it after the read is complete.

--NOTE that this SHOULDN'T call the JSON routine as that would attempt to DECODE the file.

--We want the raw (sorta, see below) file from DropBox.

function DropBox:Read(filename, callback)

local cb = callback or nil

local params = self:BuildParams()

http.get("https://api-content.dropbox.com/1/files/sandbox/".. filename .. params,

function(data,status,headers)

--return raw data - no post processing.

--NOTE THAT CODEA INTERJECTS ITSELF AND WILL PARSE AN IMAGE

--INTO THE IMAGE CLASS.

if cb ~= nil then cb(data) end

end,

httpError)

end

--searches the DropBox for a file name. At least 3 chars are needed, we don't assert it though.

function DropBox:Search(query, callback)

local params = self:BuildParams()

local searchparams = params .. "&query=" .. query

local url = "https://api.dropbox.com/1/search/sandbox".. searchparams

DropBox:_GetJSONResult(url, callback)

end

--Copies a file from one DropBox Sandbox place to another - NOT USING FILEREF.

function DropBox:Copy(src, dest, callback)

local params = self:BuildParams()

local copyparams = params .. "&root=sandbox&from_path=" .. src .. "&to_path=" ..dest

local url = "https://api.dropbox.com/1/fileops/copy" .. copyparams

DropBox:_GetJSONResult(url, callback)

end

--Moves a file from one DropBox Sandbox place to another - NOT USING FILEREF.

function DropBox:Move(src, dest, callback)

local params = self:BuildParams()

local moveparams = params .. "&root=sandbox&from_path=" .. src .. "&to_path=" ..dest

local url = "https://api.dropbox.com/1/fileops/move" .. moveparams

DropBox:_GetJSONResult(url, callback)

end

--Deletes a Dropbox Sandbox file. DOES NOT ASK FOR APPROVAL.

function DropBox:Delete(filename, callback)

local params = self:BuildParams()

local deleteparams = params .. "&root=sandbox&path=" ..filename

local url = "https://api.dropbox.com/1/fileops/delete" .. deleteparams

DropBox:_GetJSONResult(url, callback)

end

function DropBox:CreateFolder(foldername, callback)

local params = self:BuildParams()

local folderparams = params .. "&root=sandbox&path=" ..foldername

local url = "https://api.dropbox.com/1/fileops/create_folder" .. folderparams

DropBox:_GetJSONResult(url, callback)

end

~~~

 

Test Harness

I wrote this scary test harness code to validate that my DropBox is working, and so that I have examples of you to use it effectively. For example, creating a directory and them putting a file in it is not as simple as it seems. You can look through the code and pick the features you need. Sometimes a function below will fail; it's usually because we don't allow DropBox enough time to store data. This can especially happen with directories.

~~~

--users can remove this function when integrating - it's proof of operation.

--Note that since we are doing callbacks, things to not generate output in sequence;

--some operations take longer and shorter than others.

function DropBox:TestHarness(callback)

--report my "info"

self:GetInfo(callback)

--write an arbtrary byte file, and test the "zeroes" issue we get with Codea.

output = string.char(65,0,0,126)

fname = "test.bin"

self:Write(fname, output,

function(data) print("File Written") end)

--read the file

self:Read(fname, function(data) print("File Read") end)

--delete and create folders. Cascade. we really should check the JSON for an error node!

--we are intentionally forcing one thing to wait for another this way.

self:Delete("/testfolderfromcodea2",

function(data)

print("Directory deleted")

--after the delete is complete, call the create!

self:CreateFolder("/testfolderfromcodea2",

function(data)

print("Directory added")

end)

end)

--we really should nest this into the above function, but that would be hard to

--read and understand.

--copy the earlier created "test.bin" file into the new folder we just created.

self:Copy("/test.bin", "/testfolderfromcodea2/test.bin",

function(data) print ("File Copied")

--once copied, move the file from the subfolder to the top level dir, and rename it

self:Move("/testfolderfromcodea2/test.bin", "/moved.bin",

function(data) print ("File Moved")

end)

end)

--last call, searches the sandbox for the file we just moved.

print("Search Results for moved")

self:Search("moved", callback)

end

~~

Allowing other Codea projects to use DropBox

Simply add your DropBox library and the JSON library to any other app. Call the functions as you need to!