PLASTK version 0.1

PLASTK GUI ScreenshotIn a departure from my recent blog themes, I’d like to get back to AI and machine learning today to announce the first development release of PLASTK: The Python Learning Agent Software Toolkit.

From the PLASTK README.txt file:

PLASTK is a Python class library for building and experimenting with learning agents, i.e. software programs that interact in a closed loop with an environment (agents) and that learn. Especially, but not limited to, reinforcement learning agents.
PLASTK has an extensible component architecture for defining the components that make up a software agent system, including agents, and environments, and various components needed to build agents, such as function approximators and vector quantizers.

PLASTK Pendulum DemoPLASTK started life as an ad-hoc collection of algorithms from the reinforcement learning and self-organization literature that I implemented for my dissertation experiments. While in grad school I managed to clean it up to the point where a couple of other students were able to use it. The current release is the first step in my effort to make it usable outside a close circle of labmates.

PLASTK currently contains implementations of Q-learning and Sarsa agents tabular state and linear feature representations, self-organizing (Kohonen) maps, growing neural gas, linear, affine, and locally weighted regression. It also contains some demo environments including a two dimensional “gridworld” (shown in the figure), and a pendulum. Included examples show how to set up agents to learn in the gridworld and pendulum environments. PLASTK also includes a simple, component-based GUI, shown in the screenshot on the right, for visualizing agent/environment interaction.

PLASTK’s biggest deficit right now is that the agent/environment interaction API is the long-deprecated RLI version 5, from Rich Suttons RLAI group. One of the first to-do-list items is to update the interface to the new RL-Glue standard.

The PLASTK code is redistributable under GPL v.2, and available for download on Please feel free to contribute! Patches are welcome.

Helping the Stars Align for Entrepreneurs

In my recent posts about the role of big companies in developing a local start-up economy I said:

People often ask why Pittsburgh, with it’s great universities, isn’t seeding more startups. Maybe it’s because an essential missing component is a large pool of large tech companies to employ university engineers and scientists after graduation, until the stars align properly for them to start their own ventures. [emphasis added]

So what does it mean for the stars to align properly? I would say it means that it requires a confluence of several things, including, but not necessarily limited to:

    1. A good idea. (Though this may not be as important as it’s often made out to be.)
    2. The coming together of set of potential co-founders with complementary skills and highly compatible personalities. (It seems that few tech startups start out as solo operations.)
    3. One or more of the founders having the freedom to work on the project full time.

      I’d like to focus on point 3. The stereotype of a tech startup founder is a 20-something dude who is essentially the equivalent of a graduate student. Either an actual grad-school dropout (see Google) or someone who is in essentially the same position: young, single, no kids, no mortgage. Essentially, someone who is already inured to poverty by virtue of having recently been a student, and who is in a position to work long hours for almost no money in return for the chance at a big payoff sometime in the future that has a low probability of actually occurring. (see? like a grad student!)

      This stereotype exists primarily out of necessity. Start-ups usually run on a shoestring at the beginning, and a certain amount of work has to be done essentially for free. Even when seed money is available, it’s usually not much. InnovationWorks’ AlphaLab provides $25k for six months, plus free office space and support. Y combinator provides $5k per founder + $5k. None of these places provide health insurance, of course. The combination of low initial funding and lack of health insurance makes starting such a company much more difficult anyone with kids, or a mortgage or other debt burden (possibly including heavy student loans?). The economics of start-ups biases the start-up world strongly against people in their 30’s and 40’s.

      Some will say that this is all as it should be. Young people, after all, are in the best position to accept risk. There’s no reason, however, that conditions 1 & 2 above should apply only to young people. People of all ages can have good ideas, and people who’ve been working for a while are maybe more likely to have built a good network in which they can find a good set of co-founders. So if a city or region like Pittsburgh wants to encourage entrepreneurship, a good place to start is point 3 above, i.e. to work on arranging the local economy to give people the opportunity to step into entrepreneurship. Below are some suggested ways to start. Some of these probably require local or state government involvement, others might be addressed by some of the local groups trying to help support local entrepreneurship.

      • Attract and keep large tech companies that can provide high-paying jobs to potential tech entrepreneurs. A strong tech economy will allow smart people to build up the personal capital needed to make the jump to self-employment, and will also provide a soft place for them to land for those whose ventures fail.
      • Pump up the local job market generally, allowing the spouses/partners of potential entrepreneurs to get full-time jobs with health insurance.
      • Work to keep the cost-of-living down. Economic forces put this point somewhat at odds with the previous two points, but Pittsburgh already has a leg up, here.
      • Create and promote an reasonably priced, high quality group health plan for the self-employed. Pennsylvania CHIP covers all kids, and COBRA can cover adults for a few months after they make the leap, but a well advertised comprehensive group plan would be better. If UPMC wanted to contribute to the local economy, this is one place where they could start. Alternatively, the plethora of local tech-help nonprofits could band together to create one.
      • Create a culture that lauds and praises the risk-taking entrepreneur. This one is admittedly harder. It’s easy to lament the difficulty of changing Pittsburgh’s stodgy culture. Still, I think there’s room for innovative approaches here. Some kind of well-publicized prizes, or at least press features on local entrepreneurship figures? Pop City‘s Pop Star series attempts something like this. The Pop Stars are not all tech entrepreneurs, however, and in any case, Pop City’s uncritical coverage of everything borders on obsequiousness, diminishing the strength of their endorsements. Another idea: maybe one of the local philanthropic groups should spend some of their money to hire a top-quality national PR firm, and charge them to make national celebrities out of a few local tech entrepreneurs.

      That final bullet doesn’t materially do anything to help potential start-up founders find the freedom to make the leap, but it may give them the moral support and encouragement they need to find their own solutions, or to lower their own internal requirements for how much support they believe is necessary. I’d love to hear other suggestions.