| 00:00.31 | *** join/#storm wallflower (n=wallflow@ip-205-246-113-216.pool.grokthis.net) |
| 01:27.23 | *** join/#storm thumper (n=tim@canonical/launchpad/thumper) |
| 01:32.11 | *** join/#storm thumper_laptop (n=tim@125-236-193-95.adsl.xtra.co.nz) |
| 03:24.10 | *** join/#storm rockstar_ (n=rockstar@75-171-134-187.hlrn.qwest.net) |
| 04:52.43 | *** join/#storm dobee (n=dobee@85-124-200-100.static.xdsl-line.inode.at) |
| 05:33.10 | *** join/#storm dobee (n=dobee@lsfw01.lovelysystems.com) |
| 05:40.21 | *** join/#storm bac` (n=bac@cpe-065-190-187-178.nc.res.rr.com) |
| 06:00.29 | *** join/#storm jukart (n=jukart@lsfw01.lovelysystems.com) |
| 06:04.02 | *** join/#storm bac (n=bac@cpe-065-190-187-178.nc.res.rr.com) |
| 08:37.51 | *** join/#storm mcella (n=mcella@ip-176-247.sn2.eutelia.it) |
| 08:44.33 | *** join/#storm jkakar (n=jkakar@nat/canonical/x-9a8889ffcbf16be9) |
| 08:46.06 | *** join/#storm niemeyer (n=niemeyer@nat/canonical/x-1919aa3dfc385dee) |
| 08:46.33 | niemeyer | Morning! |
| 08:48.30 | jamesh | hi niemeyer |
| 08:48.42 | niemeyer | jamesh: Yo! |
| 08:49.05 | jamesh | do you have any ultra fast C extensions for me? :) |
| 08:49.13 | niemeyer | jamesh: I didn't manage to finish the cextensions work yet, but I've made some good progress |
| 08:49.19 | jamesh | cool |
| 08:49.54 | niemeyer | jamesh: I fixed what was broken, and am halfway through the Compile implementation |
| 08:50.11 | jamesh | niemeyer: at least in the short term, I am replacing the Cache() implementation for LP with one that doesn't evict |
| 08:50.30 | jamesh | essentially the same effect as the normal cache with a sys.maxint size |
| 08:50.39 | jamesh | but without overhead of maintaining the LRU list |
| 08:52.27 | niemeyer | jamesh: Hmm, interesting |
| 08:53.03 | jamesh | niemeyer: eventually it'd be nice to have an interface to ZStorm (perhaps in ZCML) to set this policy |
| 08:53.05 | niemeyer | jamesh: Do you think the overhead is the issue, or the fact that it doesn't keep items forever? |
| 08:53.09 | jamesh | but for now I'm just monkey patching it in |
| 08:53.52 | niemeyer | jamesh: Definitely.. it should be fully configurable eventually.. we just didn't want to make any promises about API and whatnot on that first god |
| 08:53.56 | jamesh | niemeyer: unfortunately, we've got code that expects objects not to expire within the transaction |
| 08:53.57 | niemeyer | s/god/go |
| 08:54.12 | niemeyer | jamesh: Oh, wow |
| 08:54.32 | niemeyer | jamesh: Do you think the non-expiry could be an issue somewhere else? |
| 08:54.38 | jamesh | I mean "not be garbage collected" |
| 08:54.56 | niemeyer | jamesh: Like, iterating a very large number of objects, or is that never done in Launchpad? |
| 08:56.17 | jamesh | niemeyer: it definitely isn't optimal, but we had a few important pages time out when putting this code live on edge |
| 08:56.33 | jamesh | niemeyer: so at the moment I'm trying to cut overhead where possible |
| 08:57.10 | niemeyer | jamesh: I see.. was just reading the message from Francis |
| 08:57.13 | jamesh | and the old code worked with "cache forever within the transaction", so it isn't introducing any problems we didn't already have ... |
| 08:59.52 | niemeyer | jamesh: Gotcha |
| 09:03.24 | jamesh | niemeyer: when you think you have something worth testing (even if it isn't completely finished), I can give it a spin locally |
| 09:04.19 | thumper | niemeyer: can I get you to review therve's branch with group_by? |
| 09:04.25 | thumper | niemeyer: I actually want to use that :) |
| 09:04.36 | niemeyer | therve: Hey! |
| 09:04.41 | niemeyer | thumper: I mean, hey! :) |
| 09:04.51 | niemeyer | thumper: Definitely! |
| 09:04.54 | thumper | niemeyer: howdy |
| 09:04.59 | niemeyer | thumper: I'll have a look at that today then |
| 09:05.06 | thumper | niemeyer: I'm looking forward to getting you guys using the new code review features |
| 09:05.25 | thumper | niemeyer: though best to wait for the 1.2.6 release of LP so you can reply to the sent email |
| 09:05.41 | niemeyer | thumper: Oh, nice! |
| 09:12.32 | *** join/#storm mcella (n=mcella@ip-176-247.sn2.eutelia.it) |
| 09:18.30 | *** join/#storm ubot3 (n=ubot3@unaffiliated/nalioth/bot/ubot3) |
| 14:54.34 | *** join/#storm nullie (n=ilya@87.254.134.62) |
| 14:54.40 | nullie | hello |
| 14:58.26 | nullie | I used to attach files to sqlalchemy entities by adding hooks for insert and delete, so I could create file attached to id after entity got to database and remove file before entity gets deleted from database |
| 14:58.40 | nullie | Can I do that with storm? I can't find appropriate hooks |
| 15:00.27 | radix | nullie: what's the SQL feature you want to use? |
| 15:00.31 | jkakar | nullie: Do you mean you write a file to the disk on INSERT (of some object) and remove it on DELETE? |
| 15:00.43 | nullie | jkakar, yes, after INSERT and before DELETE |
| 15:01.10 | nullie | radix, that's not sql feature, that's mapper feature. I need after_insert and before_delete hooks |
| 15:01.44 | radix | nullie: isn't it better to do that after committing the transaction? |
| 15:01.49 | radix | since that's what represents the actual change in the database? |
| 15:01.57 | radix | and otherwise you could have inconsistency |
| 15:02.09 | jkakar | nullie: Are you worried about changes made to the database behind Storm's back? |
| 15:03.02 | jkakar | nullie: Could you not do this in whatever method you use to insert or delete given objects? Or are you use store.find().remove() for the same thing in multiple places? |
| 15:03.16 | nullie | radix, humm, yes |
| 15:03.55 | nullie | jkakar, no, I'm not worried |
| 15:05.33 | nullie | jkakar, well, if I delete parent object and storm deletes it's relations, I'd like to have attached files deleted as well |
| 15:06.15 | jkakar | nullie: Hmm. Storm doesn't do cascading deletes, so this would all happen in the database... Storm wouldn't know to broadcast events for chlid relations being deleted. |
| 15:06.19 | radix | nullie: if by that last thing you mean ON DELETE CASCADE, storm isn't the thing that does the deleting |
| 15:06.22 | nullie | humm |
| 15:06.29 | radix | it's the database |
| 15:06.45 | nullie | well |
| 15:07.43 | nullie | Looks like I still haven't found right solution for that problem |
| 15:07.59 | jkakar | Can you explain more about the problem? |
| 15:08.13 | niemeyer | nullie: Another option is adding __storm_flushed__() to your object, and inside it checking if Store.of(self) is None, which would indicate that the object has been removed |
| 15:08.16 | nullie | I want to have pictures attached to some of entities |
| 15:08.32 | nullie | and don't want to store them in database |
| 15:08.50 | niemeyer | nullie: It's still not 100% safe, though, as if the transaction fails after this specific object is flushed, the database state won't reflect your filesystem attachments |
| 15:09.46 | niemeyer | nullie: On the other hand, nothing besides "post mortem" consistency checking will help you entirely in this case |
| 15:09.56 | radix | nullie: hmm, ok, you should probably just write the files into a filestore immediately and refer to their filenames in the database, no? |
| 15:10.00 | niemeyer | nullie: Two-phase committing is hard |
| 15:10.16 | nullie | probably I will resort to post mortem checking |
| 15:10.24 | radix | then of course if something bad happens or the object needs to be deleted, you will need to "garbage collect" the filestore |
| 15:10.32 | nullie | still I need it for my current models |
| 15:10.43 | nullie | yes, garbage collection |
| 15:10.52 | nullie | sounds nice |
| 15:10.59 | niemeyer | Right |
| 15:14.55 | nullie | Also I wanted to ask: will [offset:limit] slicing would do the right thing on resultset? |
| 15:15.07 | nullie | uh, I mean [offset:offset+limit] |
| 15:15.56 | jkakar | You could use a trigger... copy the file to a standard location (/tmp/incoming-photos), attach the filename to a column on your entity, and then have a trigger copy it into the right place, or delete it. |
| 15:16.14 | jkakar | plpython! |
| 15:17.24 | radix | don't use triggers kids, stay in school |
| 15:17.26 | jkakar | That's also going to cause inconsistency problems though. |
| 15:17.40 | radix | nullie: yes, it does the right thing (i.e., uses OFFSET and LIMIT) |
| 15:17.51 | nullie | jkakar, that would introduce complex iterations betweeb DBMS and app |
| 15:17.55 | nullie | interactions |
| 15:18.03 | jkakar | nullie: Yeah. Trigger are terrible. :) |
| 15:18.06 | jkakar | TriggerS |
| 15:18.14 | nullie | I try to follow KISS principle |
| 15:19.54 | nullie | thank you very much |
| 15:39.49 | radix | you're welcome nullie |
| 16:57.59 | *** join/#storm goschtl (n=goschtl@80.187.146.122) |
| 17:02.28 | goschtl | hi, i ve seen that there is a group_by branch. any plans to add this to a new release? |
| 17:04.19 | therve | goschtl: it will in trunk 'real soon now' (tm) |
| 17:04.40 | therve | then the next release should be done, which could take some time |
| 17:04.43 | goschtl | therve: cool how can i download the branch? |
| 17:05.08 | therve | well it should be specified in launchpad? |
| 17:05.47 | therve | something like bzr branch lp:~therve/storm/resultset-groupby-2 I think |
| 17:15.05 | goschtl | therve: i have found it thanks |
| 18:21.51 | *** join/#storm bigdog (n=scmikes@72-197-8-8-arpa.cust.cinci.current.net) |
| 18:36.33 | *** join/#storm ubot3 (n=ubot3@unaffiliated/nalioth/bot/ubot3) |
| 18:46.42 | nullie | in current release, is it possible to express SELECT t2.count(*), t1.id FROM t1, t2 WHERE t2.parent_id == t1.id GROUP BY t1.id |
| 19:29.34 | jkakar | nullie: Nope. therve has a branch at lp:~therve/storm/resultset-groupby-2 in review that introduces the ability to do this. It's in review and wlil be in the next release. |
| 19:30.01 | jkakar | We're planning on doing the release relatively soon, but it likely be a few weeks yet. |
| 19:30.52 | therve | you can do it using a Select class, but not using store.find(...) |
| 19:31.55 | nullie | ok |