Revised 6/21/2015
1. Introduction
The purpose of this document is to describe the intended code “workflow” that our AI developers will follow when they intend to contribute their code back to the main MIT repository.
All code that becomes part of the MIT App Inventor System must be tested by the developers working on the code and all code must be reviewed and tested by an experienced MIT developer prior to being accepted into the official MIT App Inventor source code repository and placed into production by MIT.
The git software itself does not include a code review support component. However the Github website where we are hosting our code offers an integrated code review system. We will discuss how we use this system in this document.
git is a distributed source code repository systems. Each contributor will usually have a copy of the entire code repository, including all changes and history. A developer will start with an up-to-date copy of the code repository and “commit” changes to this local copy. Those changes are then either “pushed” up to the parent repository or “pulled” from the parent.
A good introduction on common “workflows” by doing “man gitworkflows” on a Unix or MacOSX system with git installed. Another source of information on git may be found at http://git-scm.com/doc. and Understanding Git Conceptually by Charles Duan.
1.1 Glossary and Basics
Once you get started you will be working with two copies of the App Inventor source. We’ll define names for them here and use those names in this document.
1.1.1 The main or “upstream” repository
This is the official source code repository for MIT App Inventor. It resides in GitHub at:
Only changes for review are entered into this repository as pull request explained later in this document.
1.1.2 Your personal “FORK” repository
Each developer has their own copy of the repository located at GitHub. You create it by selecting the “Fork” button on the main repository. You will use this repository to stage changes that are destined for the main repository, or simply to keep your own code that you do not plan to contribute back. You can create branches and generally do what you want with this repository. It effectively belongs to you, the developer. You can also destroy and recreate this repository if you wish. It is located at:
git@github:<your-user-name-at-github>/appinventor-sources.git
(once you create it).
1.1.3 Your Local Repository and Working directory
Your local repository resides on your own computer. You create it by cloning your FORK repository. You will stage changes to your FORK repository here in your local repository.
As part of your local repository, when you clone a git repository, not only is the repository itself copied, but a copy of the most recent version of the “master” branch is placed in the working directory. After a clone, you will likely see the working directory. The local repository is located in a subdirectory of the working directory named “.git”. The contents of .git are manipulated by the git program and are not generally readable by humans.
Git repositories may have multiple “branches.” The primary branch is named “master” (as opposed to “trunk” in other source control solutions). You create a working directory by checking out a branch. Initially after your repository is cloned, an implicit “git checkout master” was done.
2. Getting Started
The first step to accessing and using the MIT App Inventor source code is to get an account at github.com. Click on the “Plans/Pricing and Signup” button. Signup for a free (open source) account.
You should also upload an “SSH” public key to your account (see “Account Settings”). If you do not have an SSH key, you will need to create one. Follow this link for details on ssh.
GitHub itself contains a lot of documentation. A good place to start is http://help.github.com/mac-set-up-git/ (this page is oriented to Mac users, but there are prominent links right at the beginning pointing to Linux and Windows versions).
2.1 Fork the “upstream” repository
Navigate to: https://github.com/mit-cml/appinventor-sources
Once selected you will see a row of buttons near the top. Select the “Fork” button.
This will create a your personal FORK repository on github.com. This is a copy of the main repository. This is called a “fork” of the main repository.
You can use SSH or HTTPS to clone the git repository onto your machine. In the following instructions, SSH commands will be followed by HTTPS commands. Only complete one set (SSH or HTTPS) of commands!
You now need to create the local repository on your own computer. Connect to a directory on your computer where you want to keep your App Inventor work. Now open a command window and run (replace USER by your user name):
OR
git clone https://github.com/USER/appinventor-sources.git
Next, do:
cd appinventor-sources
git remote -v
You should see:
origin git@github.com:USER/appinventor-sources.git (fetch)
origin git@github.com:USER/appinventor-sources.git (push)
OR
origin https://github.com/USER/appinventor-sources.git (fetch)
origin https://github.com/USER/appinventor-sources.git (push)
USER should be your Github username. Your fork repository is now the “origin” of your local clone. Be aware that your fork will not be automatically updated as the main (upstream) repository is updated with other people’s work. When you submit your work for review, you will be required to have first tested your changes with the latest App Inventor official sources from the upstream repository.
In order to update your local repository with changes from the main (upstream) repository you need to add it as an additional “remote” repository. Do this from your appinventor-sources directory:
OR
git remote add upstream https://github.com/mit-cml/appinventor-sources.git
If you do a “git remote -v” You should now see something like:
origin git@github.com:USER/appinventor-sources.git (fetch)
origin git@github.com:USER/appinventor-sources.git (push)
upstream git@github.com:mit-cml/appinventor-sources.git (fetch)
upstream git@github.com:mit-cml/appinventor-sources.git (push)
OR
origin https://github.com/USER/appinventor-sources.git (fetch)
origin https://github.com/USER/appinventor-sources.git (push)
upstream https://github.com/mit-cml/appinventor-sources.git (fetch)
upstream https://github.com/mit-cml/appinventor-sources.git (push)
(again, USER will be your Github username).
2.2 The .gitignore file
After cloning, you should find a file in the top folder called ‘sample-.gitignore’. This file should be renamed to simply ‘.gitignore’. Note that after renaming it, the file will probably be hidden from view in Unix type systems.
As it can be read in its man page, this file specifies files that are not intended to be tracked. For example, if you use an editor that creates certain types of temporary files (for instance file*.swp using vim), you can add the extension of the file to .gitignore. Any additional folders that should not be tracked, should also be added there.
3. Normal Work Flow
The local repository on your own computer has a working directory associated with it. During normal development you will create a new topic-based branch, checkout that branch and do your work on that branch.
3.1 Getting ready to make a change
We’ll presume that you have a change to make to App Inventor. Start by ensuring that your fork repository’s master branch and the local clone on your computer are both up-to-date with the upstream repository.
Do the following:
git checkout master ← Work on the master branch
git pull --ff-only origin master ← Update local clone from your fork’s master branch
git pull --ff-only upstream master ← Update local clone from the upstream repository
git push origin master ← Update your fork repository’s master branch
git pull --ff-only origin master ← Update local clone from your fork’s master branch
git pull --ff-only upstream master ← Update local clone from the upstream repository
git push origin master ← Update your fork repository’s master branch
You should also get the tags in the main repository, as they are used to keep in sync with the build server:
git fetch --tags upstream master
Now pick a name for a “topic” or “feature” branch that you will create for you to change. Let’s say you are adding the “foo” component. You might then name your branch “foocomp” (it doesn’t really matter what name you pick, but using names that make sense to you is the best bet). You create the branch with:
git checkout -b foocomp
This will create a new branch named “foocomp” off of the master branch. Your working directory will now be associated with the foocomp branch. You can verify this by using
git branch
to view your branches, and you should see something like:
* foocomp
master
(Note that ‘git checkout -b foocomp’ would be the equivalent to doing ‘git branch foocomp; git checkout foocomp’)
In general you can have lots of branches in git that relate in a tree structure. That’s a great way to get confused, so try to stay with just the master branch and a branch for the change you’re working on. If you need to work on several changes at once, you can create a separate branch for each change, but it is very easy to get confused if you’re switching between those branches in the same local clone of the repository. It is less error-prone to work on the different branches in different local clones of your fork.
3.2 Making changes. staging, and committing
You can now do your code development, making your changes to the working directory. As you work, it’s a good idea to periodically commit your changes. A commit creates a checkpoint in your local repository. That’s useful, because you can back up to a previous commit point if you get fouled up as you are working. You can also ask git to show you the differences between your current state and a given commit point, and other useful and confusing things. See http://learn.github.com/p/normal.html for this, and more.
You do a commit with the “git commit” command and a brief message describing the commit, for example
git commit -m “I finished the first method for this awesome component”
Warning: Commits occur purely on your local machine, not on your fork on GitHub. We’ll deal with pushing things to GitHub below.
One tricky property of git is that changes must be “staged” before they can be committed. If you modify a bunch of files and then commit, nothing will be committed until you explicitly “git add” them to the list of files to be committed. You can keep track of all this with the aid of the
git status
command. It will show you a list of “changes to be committed”, i.e., changes that will become part of the next commit. It will also show you a list of things that are “changed but not updated” - things that must be staged in order to be added to “changes to be committed”. You can stage these by using “git add” on them one at a time, or you can use
git add -u :/
to stage all the file modifications and deletions. You can also cause the staging to happen automatically, by using git commit with the -a flag:
git commit -a -m ‘Here is the next method for the awesome component’
Note: if you intend to use ‘-u’, make sure your .gitignore file is set properly (as explained in section 2.2), otherwise some unwanted files might end up in your commit.
Warning: If you use checkout to switch to another branch, your uncommitted changes will be lost. That’s a good reason to do periodic commits. Of course, you might actually want to throw away the uncommitted changes.
Note: If you create a new file as part of your coding, you need to explicitly “git add” it in order to get it “tracked” by git. The git status command will show you such files as “changes to be committed”. You’ll also see a bunch of files listed that you did not create, including the build files that are generated as part of the ant build process. Do not add these to git to be staged. They are not part of the App Inventor source.
3.3 Testing; Supporting older Android versions
Test your code thoroughly before submitting it for review. This includes creating a companion and trying it, and also uploading your code to appspot and making sure it works from there, not just from a server on your local machine.
Testing also includes checking that your code supports older versions of the Android operating system, back to system 2.3 (Gingerbread). If you don’t have a device running the old OS, you can use the Android AVD manager to create an emulator running the old system and test with that: Start the emulator on your machine, and tell App Inventor to connect using USB. The emulator should connect as “USB” even though there’s no USB cable.
NOTE: If your code uses features that are not supported in old operating systems, you will have to check system version numbers and implement alternate code paths that provide graceful degradation. You can find examples of this in the App Inventor source code.
3.4 Requesting code reviews
OK, so now you’ve done some work and want to get it reviewed and submitted so it can be added to the main repository of App Inventor sources.
NOTE: Don’t even think about doing this unless you have thoroughly tested your work, and also run the tests with “ant tests”. Make sure to test your work by building an apk, not just test in development mode. Also test your work when you deploy it to App Engine, as well as when you run on your local machine. And provide reviewers with the URL of an instance running your new code, so they can try it for themselves.
And in almost all cases, you should create unit tests that are included with your pull request.
NOTE2: There are a number of JavaScript tests that rely on having phantomJS in your PATH. More information about these tests can be found in this README file.
And of course, as you’ve been developing, you’ve been periodically doing
git pull upstream master
to make sure you are incorporating other people’s submitted changes into your code.
When you’re finally ready, do another pull from the upstream(main) repository and run the tests again, just to make sure. If everything still seems OK, do another commit and then push your branch to your fork on GitHub with:
git push origin foocomp
where “foocomp” is replaced by whatever you called your branch. This will create the foocomp branch on your personal fork repository on github.com.
Finally, go to GitHub on the web and look at your local fork. Select the foocomp branch using the branch pulldown menu at the upper left. Then press the button at the top of the screen marked “Pull Request”. This will start a formal review, complete with incorporated patch sets.
As part of the Pull Request you will see three tabs, one to start a conversation, one with the current commits for the PR, and one with the diffs for the files changed. To make sure that someone in the team gets notified to do a review on the code, please add their handles in the comment box (conversation tab). This is called a @mention. To do so, you can use the ‘@’ symbol and start typing a user name, and github will autocomplete. The screenshot below shows how name handlers can be used.
GitHub does not automatically assign a reviewer, but after someone in the team has been notified of a new review (please use their handlers in the comment), a reviewer will be assigned to your request. If you want to get emails when the reviewer sends comments, please make sure your notifications are set correctly by clicking on your profile settings and going to the ‘Notification center’ tab, as shown in the screenshot below:
The process will probably take several cycles of comments back and forth, so do not despair!
Whenever changes are asked by one or more reviewers, you will need to push new commits to the pull request. Make sure you are working in a feature branch, and any new commits will be added automatically when you push to your fork. It’s good practice to use ‘--rebase’ when you pull from master to keep up to date with the main repo, so that your commits bubble up to the top. That makes the reviewers work a lot easier because once the review process is completed, all the commits will be squashed into one and sent over for a final review on App Inventors Gerrit review server.
When you add the pull request add are ready for it to be reviewed, add the label “status: needs review” to the pull request. Later, during the review process, the reviewer might change that to “needs reply to review”. After you address the review comments, change the label back to “needs review”. There might be several back-and-forth cycles during the review.
When the review is complete and everything is approved, your work will be merged into the App Inventor master branch by one of the repository maintainers. Do not do the merge yourself.
Note: In more detail, your finished and approved code will be moved to another server, called a Gerrit server, and pushed to the App Inventor master branch from there. In even more detail, there are two branches on Gerrit, one for changes that will require updates to the companion: the ucr or “upcoming component release” branch, and the ordinary master branch. Code that requires updating the App Inventor Companion is merged into the ucr branch, and merges from ucr to master are batched, in order to to minimize the number of public releases that require App Inventor users to update their Companions. Code that does not require changes to the companion is merged into the master branch right away. You don’t have to worry about any of this as a developer: just work on your local repository and fork and your master branch, and the repository maintainers will deal with Gerrit and ucr.
3.5 Cleaning up after a change and otherwise updating your fork and local repositories
After your changes are accepted into the upstream (main) repository, you will want to download them to your fork and local repository. You have them on the foocomp branch, but they now reside in the main repository on the master branch. Your foocomp branch is effectively a vestige that exists now only in your fork and local repositories (the actual code changes you made are on the master branch).
This next part is a little counter-intuitive, as there is no way to “pull” from the main repository directly to your fork. Instead you pull from the main repository to your local repository and then push to your fork. This is why we setup the upstream (main) remote in section 2.1.
First, checkout the latest version of the master branch (do this even if you are using the same local directory where you made your changes that will be pulled in):
git checkout master
Now do a:
git pull --ff-only upstream master
This should pull your changes (and any other changes) from the main repository to your local repository and place them on the master branch. Now you can:
git push origin master
This will place those changes onto the master branch of your fork repository.
3.5.1 Getting rid of your development branch
Now if you are done with your foocomp branch, and all its valid changes are on the master branch, you can get rid of foocomp. Do this:
git branch -D foocomp
This will remove it from your local repository. Now do:
git push origin :foocomp
Note the “:”, this tells git you are removing the foocomp branch from “origin” (which is your fork repository).
More information and details can be found in the GitHub documentation at http://help.github.com/send-pull-requests/.
3.6 Quick guide for github and dev workflow
The following slide decks can be used as quick guides for Contributing to App Inventor 2 and App Inventor 2 development workflow.
Unless you are familiar with github, you won’t be able to simply follow the slide decks without reading this document, so please do read this one first, and then use the quick guides as refreshers.
3.7 Notes for Code Reviewers
3.7.1 Testing the changes
If you are asked to do a code review you might want to build and test the changes yourself (especially if they are extensive). Here is one way to do that in git.
You can directly access the code of a Pull Request by fetching a special branch name.
git fetch upstream pull/<Pull Request Number>/head
For example to fetch the code for Pull Request 77 you would:
git fetch upstream pull/77/head
This places the code in the special branch “FETCH_HEAD”. You can then checkout this branch, or give it a local branch name. For example:
git branch TESTFIX FETCH_HEAD
Will make the branch TESTFIX in your local repository based on FETCH_HEAD. You can now build the code from the Pull Request, compare it to other code or whatever else you need to do to perform your code review.
When you are satisfied with a change, add a comment to the Pull Request which contains the string “LGTM” (short for “Looks Good To Me”). Then add the label “pull request: ready to push to gerrit.”
At this point one of the repository maintainers will squash the Pull Request (if needed) and submit it to Gerrit for final merging into the master branch.
4. Some tips and tricks for speeding up development
This section contains a few tricks that some of us use for speeding up App Inventor when we are full on development mode. Note that most of these tricks should only be used while developing, but when you are ready to open a pull request you should skip all tricks, make a full compilation for all languages and browsers, and fully test (manually as much as you can, and always run `ant tests`).
The first thing you should check (at least from time to time) is the listing of available targets in the main ant script. Here is how:
Tip 1: check out all targets available with `ant -p` from time to time.
At this particular time, it looks like the following:
appinventor-jos$ ant -p
Buildfile: ~/appinventor/build.xml
Other targets:
AIMergerApp
Blockly
BlocklyTest
PlayApp
RunLocalBuildServer
all
clean
comps
installplay
javadoc
noplay
tests
Default target: all
|
As you can see, there are a number of targets here, but in this post I will only go into the ones I consider that save time during development. The following are only recommendations, you do not have to do this, of course!
Also consider that if you see a new target being added, make sure you understand how to use it because it can save you some time in the long run.
Tip 2: avoid using `ant clean` whenever you can.
Compiling App Inventor is done incrementally, so only what's needed to be compiled will be compiled. Unfortunately, the GWT part of the system takes literally ages when a full compilation needs to happen (see Tip 6 for more on this). If you are making changes in that part of the system, you are kind of stuck with a full compilation, but it does not mean you need to clean all the time.
The clean target does not only delete all the compiled files, but also deletes all your projects, so use with caution. When would you use clean, then? Well, if things start to get weird, like rare errors you haven't seen before, or compilation errors that do not seem to appear in your IDE, then it's time to clean. Another good indication of cleaning time is if you change branches that contain incompatible code. But if you can avoid this, there are a number of other targets that can help you out. Let's see some of them.
Tip 3: When in Blocky Land, do as the Blocklians do.
All the App Inventor Blockly related code gets compiled into a single file called blockly-all.js that you will find in your build folder. If you are making JavaScript changes (and only in the /blocklyeditor or lib/blockly files) you can rebuild this file by calling `ant Blockly` (note the Capital B). No other files need rebuilding (generally), and this target takes time in the order of seconds and not minutes! Make sure you check out tip 4 to see how to reload this file without restarting the main server.
Tip 4: There's no need to restart the main server --- most of the time!
Most times there is no need to restart the app engine server. By disabling caching in your browser(s) and reloading the project, all should be good to go.
You can disable caching in Chrome Dev tools or in Firefox (probably in Safari too!).
In Firefox, go to 'Tools --> Web Developer --> Developer Toolbar' and click on the wrench icon at the end of the toolbar, at the bottom of the page. The toolbar will enlarge and then you can click on a gear icon for settings. Scroll down to 'Advanced Settings' and you will see an option like the one in the following figure:
 |
Disabling caching -- Firefox
|
In Chrome Dev Tools you simply need to click on the gear icon for settings and you will see something similar to the next figure:
 |
Disabling caching -- Google Chrome
|
Note that disabling cache only applies when you have the dev tools open, so you don't have to worry about your normal web surfing with this setting.
Tip 5: Component changes might not need a full build
UNLESS you change the signature of a method (in blocks this can be a method or event signature, or a property), you will not need to reflect those changes in the designer, so there's no need to build the GWT part of the system.
When no changes to the designer are needed, you can use `ant comps`, which will not only compile exclusively changes to your component, but will also push a freshly baked App Inventor Companion app to your phone if you have it connected through USB.
Tip 6: Minimize permutations in GWT compilation
There are two ways of minimizing the number of compilations that GWT will run during a full build, one being restricting the number of browsers you build for, and the second being the number of languages you fully build. These changes can be made in the YaClient.gwt.xml file.
If you generally develop only using Chrome, or Safari, or Firefox, you can modify the GWT config file to avoid building for other browsers. In line 10 in the linked gist, in the value for user.agent you can choose between 'safari' and 'gecko1_8'. If you generally only use firefox, you can delete everything before the comma (including the comma, keeping gecko1_8). If you generally use chrome or safari, keep the safari value and delete the comma and the gecko part. This will compile GWT only for the browser you have chosen.
Since we have introduced the internationalization code, the more languages we add, the longer it takes to build (more permutations of the code). In the gist again, if you comment out lines 104 to 109 (in the gist; lines 108 to 116 in the current real file), both included, you will only build the interface in English.
You can combine languages and browser and go down to 2 permutations instead of 9, which will save several minutes from your build.
Note: Jeff Schiller has a special commit that implements the changes above. To use it do:
“git cherry-pick FETCH_HEAD”
Alternatively you can replace the last command with:
“git cherry-pick --no-commit FETCH_HEAD”
This will bring in the changes but not create a commit.
Make sure you remove or revert this commit before you submit code for review.
Tip 7: Do you really need that build server running all of the time?
This isn't really a compilation tip, but it might also save you time while developing.
Some people use a starting script that runs both the app engine server and the build server. Most of the time (note, not all of the time!) you will be making a bunch of changes and testing them on the companion if they are component changes, or in the browser if they are interface changes. You will only build an apk towards the end of the process (in component changes) before submitting a pull request. So you won't need to spin up the build server most of the time.
NOTE
As mentioned at the top of this section, please note that most of these tips are to be used only during development, BUT before you open a pull request, you should test fully, and with all languages and all browsers.
Happy (and hopefully shorter) building!
5. Debugging Kawa code
One of the difficult parts of App Inventor to debug is investigating the Kawa code in runtime.scm. About the best you can do is add statements to write messages into the Android system log, then rebuild and run the system. It would often be better and faster to evaluate expressions directly in the same Scheme interpreter (REPL) that runs in the phone. You can do this by building a modified version of App Inventor, together a modified version of the companion, that will evaluate expressions sent from your computer. Here’s how to do this:
- If you are debugging, commit your changes so far, and check out a new branch.
- On your new branch do:
- “git cherry-pick FETCH_HEAD”
- This will pull over one commit which contains the code to modify the Companion to handle arbitrary YAIL code as well as a Python script to use it. The commit message has details.
- WARNING: Do not distribute (widely) a Companion with this code, to do it’s work it introduces a security vulnerability into the Companion. It is not for production. You have been warned! :-)
- Build App inventor. It will have the code you are working on, together with the new patch. Also build a new companion and install it.
- Start App Inventor and connect to the companion. You can use App Inventor as usual.
- But you can also evaluate expressions by typing them to a driver running on the computer. To start the driver loop, run the python command python driver.py <arg> were arg is the IP address of the phone (use 127.0.0.1 if the phone in connected by USB). Note that each expression you evaluate must be a single line.
Appendix A -- SSH Keys
When you perform git operations on github.com, github recognizes and authenticates your access via SSH public/private keys. If you already use SSH you should have a directory named “.ssh” under your home directory. This directory will contain your keys and other related information. In particular the file “id_rsa.pub” contains your RSA public SSH key. This is the key that you upload to your github account (you will actually open the file in a text editor and paste the contents into a web form on github). DO NOT UPLOAD id_rsa (the version without the .pub) as that is your private key. This key should not be shared with anyone!
If you do not yet have SSH keys, the command to create them is “ssh-keygen”. If invoked without arguments it should create an RSA and DSA key for you. Once you have run ssh-keygen, you should have the id_rsa.pub file needed in the above paragraph.
댓글 없음:
댓글 쓰기