tag:blogger.com,1999:blog-8480453432599016782024-03-05T10:57:51.381-08:00Taumuon JabukaGary Evans coding blogAnonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.comBlogger42125tag:blogger.com,1999:blog-848045343259901678.post-55998347088119908292014-10-01T13:29:00.002-07:002014-10-01T13:45:56.412-07:00.NET SIMD to solve Burgers' equationI'm enrolled on the mooc <a href="http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about">Practical Numerical Methods in Python</a>, and module 2 has covered the <a href="http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_04_1DBurgers.ipynb">Burgers' Equation</a>. This is a great course by the way, and it's not too late to join up!<br />
<br />
I am impressed with the productivity of Python. ipython notebooks, SymPy, plotting etc. are all great and I feel that .NET could benefit from a similar ecosystem.<br />
<br />
That said, I've been itching to play with the new .NET SIMD support and this seemed to be a suitable example.<br />
<br />
The Python notebook demonstrated how using Python's array notation to solve the Burgers' equation using finite difference resulted in a 9 fold speed increase, on my machine taking 25ms instead of 200ms.<br />
<br />
I translated the code into C#, and it completed in 2ms, so I immediately was impressed (not surprised though - an interpreted versus (JIT) compiled language...). This wasn't going to tell me much about SSE improvements though, so I increased the number of spatial samples a hundred fold, and the number of timesteps by 10.<br />
<br />
The time was then increased to 2580ms.<br />
<br />
Installing .NET 4.5.2 and using RyuJIT CTP4 to compile and run, the running time reduced to 1880ms. RyuJIT can't arrive fast enough!<br />
<br />
I then implemented array swappping, instead of continually creating and copying arrays, to see if it would reduce pressure on the GC (Calculate2 in the gist), but it had no significant difference.<br />
<br />
I then pulled the calculation of (nu * dt / dx) into a variable (Calculation3())<br />
<br />
This resulted in a calculation time of 230ms.<br />
<br />
Then, I felt it was time to implement SIMD. The documentation is a bit scarce, but it wasn't too tricky to figure out. My machine has 128bit SSE registers, so it can add two doubles (64 bit) in a single operation.<br />
<br />
Running this resulted in a calculation time of 90ms, so slightly greater than 2x speedup. I'm not sure whether it was able to make further optimisations based on the changed structure of the code, but this is a great win. This is implemented in Calculation4() in the gist.<br />
<br />
The gist is available here: <a href="https://gist.github.com/taumuon/5d4db567c57f9846971d">https://gist.github.com/taumuon/5d4db567c57f9846971d</a>Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com1tag:blogger.com,1999:blog-848045343259901678.post-39067137395962241082014-08-25T01:57:00.002-07:002014-08-25T01:59:12.387-07:00Monte Carlo double barrier option pricing on GPU using C++ AMPI converted the double barrier option pricing code from my last post to run on the GPU, using C++ AMP, and got an 80% speedup.<br />
<br />
This is running on a low powered Acer V5-122p laptop with an AMD A6-1450 processor with an integrated Radeon HD 8250 GPU.<br />
<br />
The gist is here: <a href="https://gist.github.com/taumuon/bbeeb9e2c1f5082a2699">https://gist.github.com/taumuon/bbeeb9e2c1f5082a2699</a><br />
<br />
To be fair, I converted the CPU code to be otherwise identical to the gpu code. Instead of populating an array of the sample path, it instead for each point just determines whether the value breaches the upper or lower barriers and uses the last value for the payoff.<br />
<br />
This reduced the runtime from the code in my last blog post, of 2540ms, to 1250ms, i.e. a 2x speedup.<br />
<br />
The GPU code was ran twice, the first time it ran in 140ms, the second run (after all shaders already compiled etc) was 15.6ms, i.e. a very impressive 80x speedup from the CPU code.<br />
<br />
If anything, it shows that AMDs strategy of cheap low powered laptop chips will payoff if people start taking advantage of the relatively strong GPU.Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com2tag:blogger.com,1999:blog-848045343259901678.post-13619910256602902052014-08-04T11:47:00.002-07:002014-08-25T01:59:12.380-07:00Monte Carlo pricing of Exotic Options in Functional Style C++My last blog post was about <a href="http://taumuon-jabuka.blogspot.co.uk/2014/04/monte-carlo-pricing-of-exotic-options.html">pricing of exotic options using F#</a>. In that post I said "Apart from the Monte Carlo code, the fact that in F# all functions are
in the curried form means that composing the payoff functions using
partial function application is beautiful. This would be lambda hell to
replicate in C#."<br />
<br />
The OO version of this would probably be built via composition: having a class implementing a IPayoff interface (implemented for Call and Put), and that would be used in implementations of IPathPayof, which would be implemented by AsianArithmeticMean, DoubleBarrier etc. The use of OO is to pull out the common implementation, but there's no state associated with any of these classes - it feels much nicer using function composition. <br />
<br />
I've implemented a version in C++ using lambdas, and the functional composition is actually really nice. C++ has the std::bind method which means that any number of parameters can be partially applied even for functions not in the curried form.<br />
<br />
The code can be found here: <a href="https://gist.github.com/taumuon/727c31f4a56518f91718">https://gist.github.com/taumuon/727c31f4a56518f91718</a> <br />
<br />
The most surprising thing was the speed increase - the equivalent algorithm in C++ is 20x faster than the F# code! For instance, running on my low-powered AMD A6-1450 powered laptop, the double barrier option took 45 seconds to calculate in F#, whereas it took 2.2 seconds in C++.<br />
<br />
I did tweak the F# to instead of using Seq.unfold to create the asset path, it instead creates and mutates an array, and that reduced the running time by more than 25%.<br />
<br />
EDIT: Jack Pappas has forked and modified the F# code so that its runtime is now in the same ballpark as the C++ code. The fork is here: <a href="https://gist.github.com/jack-pappas/d629a767823ca2f17967">https://gist.github.com/jack-pappas/d629a767823ca2f17967</a> <br />
<br />
I think there's scope to speed up both the F# and C++, by introducing parallelism and investigating GPGPU, which I hope to do if I don't get sidetracked.Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-28580278120933723092014-04-25T14:00:00.001-07:002014-08-25T01:59:12.383-07:00Monte Carlo pricing of Exotic Options in F#I've recently completed all exercises for the MOOC <a href="https://iversity.org/courses/monte-carlo-methods-in-finance">Monte Carlo methods in Finance</a>, ran by the University of Madrid on Iversity. The course was excellent, both in its content and delivery.<br /><br />The course started off by refreshing probability theory (cdf, pdf etc) and quickly moved onto generating random numbers, Brownian motion and stochastic differential equations followed by the pricing of options and calculating risk.<br />The final week's homework is on the Monte Carlo calculation of a portfolio's risk, using copulas with Student's t distribution, but as that week's homework is still running I'm blogging about the penultimate week's homework.<br /><br />This homework is about pricing path dependent options, and demonstrating how using a <a href="http://en.wikipedia.org/wiki/Control_variates">control variate</a> can provide a better result for the same number of iterations (better meaning less error, i.e. with a smaller standard deviation). The homework further shows that when pricing a barrier <br />option, using an up-and-out option as the control variate produces better results than using a European option as the control variate.<br /><br />The course delivery was extremely slick. The videos were filmed specially, this wasn't a rehashing of a lecture in front of students (as seems to happen in some other online courses). The lectures were short, and to the point, managing to cram a lot of information in with zero fluff. I really appreciated this as I was watching on a daily commute.<br /><br />The assessments were by multiple choice question, but with only one attempt, so it did end up actually feeling like a challenge. The questions were a mixture of maths and coding, with the balance moving more towards coding as the weeks went on.<br />The code was delivered in Matlab, which I had to get reacquainted to after a decade break. The main downside I found is that I ended up doing a fair bit of copy-paste coding, so completed some of the homeworks in .NET instead. I personally found the coding easy, and found the maths more challenging (in a good way).<br /><br />Now, for the code - the F# code is available here: <a href="https://gist.github.com/taumuon/11302896">https://gist.github.com/taumuon/11302896</a><br /><br />Apart from the Monte Carlo code, the fact that in F# all functions are in the curried form means that composing the payoff functions using partial function application is beautiful. This would be lambda hell to replicate in C#.<br /><br />If this course runs again I'd wholeheartedly recommend it.Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-62765852469535627952014-01-24T10:03:00.003-08:002014-08-25T02:00:05.677-07:00MOOCsI've just enrolled on Iversity's Monte Carlo Methods in Finance course, and have converted some of week 1's Matlab demo code over to F# and Deedle: <a href="https://gist.github.com/taumuon/8602365">https://gist.github.com/taumuon/8602365</a><br />
<br />
I spent the latter half of last year diving into various online courses. I completed Coursera's Mathematical Methods for Quantitative Finance, and also followed along with a number of other courses to varying levels of completeness.<br />
<br />
I completed three weeks assignments of Udacity's Cuda programming course. Week three was painful due to an error in the reference code, and week 4 was crashing due to a memory exception. I was using a GPU emulator in Ubuntu, and decided that it would be easier with real hardware. I watched the remaining videos and found the parallel algorithms explanations useful.<br />
<br />
I completed the first two weeks of Coursera's Scientific Computing. These were maths exercises, which I enjoyed and that's what inspired me to do the Maths Methods for Quant Finance course. The Matlab exercises I was planning to do in F#, but left the course when other attendees were complaining that to complete the homework to the correct numerical accuracy you needed the exact version of Matlab the instructor was using, and they were unable to use Gnu Octave.<br />
<br />
It is great that there's so much free high standard material available. The fixed timescale nature of the course is a bit annoying - if work or life gets in the way one week it may make it impossible to catch up with the remainder of the course. I may get around to trying a course again next time it comes around though. Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-83067284861890758792013-08-17T07:31:00.000-07:002013-08-17T07:32:49.234-07:00RunKeeper VisualisationsMy last post described how I'm using the Twitter API to receive tweets off the live stream.<br />
<br />
Since then, I've used the API to filter for tweets containing the #runkeeper hashtag, and used that to scrape the user's activity from the RunKeeper site (including the GPS points from the user's exercise). I've stored that information in a MongoDB, which has allowed me to do some simple visualisation:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/b8njarJC6qE?feature=player_embedded' frameborder='0'></iframe></div>
<br />
The above video (best played at 720p) shows activities plotted against time of day (the sun overhead in the video indicates midday for that region). <br />
<br />
I haven't charted this to confirm, but to my eyes it looks like amount of exercise activity peaks around sunrise and sunset, with almost none at night time (which isn't really a surprise).<br />
<br />
For the curious, this is what the colour of the dots indicate:<br />
<br />
<pre style="background: #1e1e1e; color: gainsboro; font-family: Consolas; font-size: 13;"><span style="color: #569cd6;">let</span> <span style="color: white;">createSphere</span> (<span style="color: white;">latitude</span><span style="color: #b4b4b4;">:</span><span style="color: white;">float</span>) (<span style="color: white;">longitude</span><span style="color: #b4b4b4;">:</span><span style="color: white;">float</span>) (<span style="color: white;">activityType</span><span style="color: #b4b4b4;">:</span><span style="color: white;">string</span>) <span style="color: #b4b4b4;">=</span>
<span style="color: #569cd6;">let</span> <span style="color: white;">color</span> <span style="color: #b4b4b4;">=</span> <span style="color: #569cd6;">match</span> <span style="color: white;">activityType</span> <span style="color: #569cd6;">with</span>
| <span style="color: #d69d85;">"RUN"</span> <span style="color: #569cd6;">-></span> <span style="color: #d69d85;">"Red"</span>
| <span style="color: #d69d85;">"WALK"</span> <span style="color: #569cd6;">-></span> <span style="color: #d69d85;">"Green"</span>
| <span style="color: #d69d85;">"BIKE"</span> <span style="color: #569cd6;">-></span> <span style="color: #d69d85;">"Blue"</span>
| <span style="color: #d69d85;">"MOUNTAINBIKE"</span> <span style="color: #569cd6;">-></span> <span style="color: #d69d85;">"Orange"</span>
| <span style="color: #d69d85;">"SKATE"</span> <span style="color: #569cd6;">-></span> <span style="color: #d69d85;">"Yellow"</span>
| <span style="color: #d69d85;">"ROWING"</span> <span style="color: #569cd6;">-></span> <span style="color: #d69d85;">"Cyan"</span>
| <span style="color: #d69d85;">"HIKE"</span> <span style="color: #569cd6;">-></span> <span style="color: #d69d85;">"Brown"</span>
| <span style="color: #d69d85;">"OTHER"</span> <span style="color: #569cd6;">-></span> <span style="color: #d69d85;">"Black"</span>
| _ <span style="color: #569cd6;">-></span> <span style="color: #d69d85;">"Black"</span>
<span style="color: white;">String</span><span style="color: #b4b4b4;">.</span><span style="color: white;">Format</span>(<span style="color: white;">sphere</span>, <span style="color: white;">latitude</span>, <span style="color: white;">longitude</span>, <span style="color: white;">color</span>)
</pre>
<br />
On a local scale, the actual GPS traces are of interest:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3d-HZh-cShkeaAIu-zVsnlm4U1uh3fPg6c9GGCqEZ-7VjdPHlFxIpMJ81Ynl9lgvS-iUfFZRW3GklNLHUivSMINEMP6s0vpo4JFRZ5_BheVT8JoKNxPwa5Rundw2eiTO4tXRfwG7Uc40/s1600/uk2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="279" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3d-HZh-cShkeaAIu-zVsnlm4U1uh3fPg6c9GGCqEZ-7VjdPHlFxIpMJ81Ynl9lgvS-iUfFZRW3GklNLHUivSMINEMP6s0vpo4JFRZ5_BheVT8JoKNxPwa5Rundw2eiTO4tXRfwG7Uc40/s320/uk2.png" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
Activities are present in the more-populated regions of the UK. The blue and orange traces indicating cycling and mountain biking activities are clearly visible.<br />
<br />
NOTE: if you think the map looks weird, it doesn't have the usual aspect ratio - I'm not using a Mercator projection do plot the points, but am simply plotting the longitude and latitude linearly.<br />
<br />
On an even smaller scale, landmarks around London are visible:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWRKVK_qJAEb9TcUllyDEHxRqXrobIG8m4nn-K3ucZ_81p1vx8foDvNOSfgtO553E5JlJ2S1-gZIC0IJV7PFvaT-Ik-6ytmQJ4zu2WRI-l4rGj_JAv7pI9uDPxUrSasYhRMrc08lhlmCs/s1600/xxx.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="170" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWRKVK_qJAEb9TcUllyDEHxRqXrobIG8m4nn-K3ucZ_81p1vx8foDvNOSfgtO553E5JlJ2S1-gZIC0IJV7PFvaT-Ik-6ytmQJ4zu2WRI-l4rGj_JAv7pI9uDPxUrSasYhRMrc08lhlmCs/s200/xxx.png" width="200" /></a></div>
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsCHUc1l_v7eeRS_ONKXDHNVG-e_w7Kh1tlAlUoDt6nOJxBb3RRN264HZEdylsGnen3aLpdFwy2RHUrb6V3ssw_9-aS4R7G9X2OwCKUFsypw91oOn4zmD-7iXwVHY5hsEGS6_RX-MH6-o/s1600/longoogle.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="135" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsCHUc1l_v7eeRS_ONKXDHNVG-e_w7Kh1tlAlUoDt6nOJxBb3RRN264HZEdylsGnen3aLpdFwy2RHUrb6V3ssw_9-aS4R7G9X2OwCKUFsypw91oOn4zmD-7iXwVHY5hsEGS6_RX-MH6-o/s200/longoogle.png" width="200" /></a><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
The GPS data contains altitude information, so there are more interesting visualisations that could be done, including generating contour plots. Also, the above only contains three days worth of data - with a larger data set it would be possible to plot to determine whether activities peak on a weekend etc.<br />
<h3>
Implementation</h3>
<h4>
Acknowledgements: </h4>
The 3D plot of the globe was drawn using <a href="http://www.povray.org/">POV-Ray</a>. The 3D globe model was from <a href="http://grabcad.com/library/earth-globe">grabcad</a>, with the converted to a POV-Ray model using <a href="https://sites.google.com/site/poseray/">PoseRay</a>.<br />
<br />
The UK outline was obtained from the <a href="http://www.ngdc.noaa.gov/mgg/coast/">NOAA</a>.<br />
<br />
The code was implemented in F# (which was a pleasure to get back to after the C++ I've been doing recently). I did try the MongoDB.FSharp library to store my data records, but they failed to deserialise from the database. In any case, I wanted more control over the data types saved (I wanted to store the GPS data as GeoJson3DGeographicCoordinates, with the first point stored separately as a GeoJson2DGeographicCoordinates with an index on this value). I could have created .NET classes and used the BSON serializer, but it seemed more effort than writing directly to the DB (and this is about the only time I've seen the benefit of C# implicit conversions, but I can live without them in F#).<br />
<br />
Why use the Twitter API, and why not scrape the RunKeeper site directly? That's because of times - the RunKeeper website displays the activity time, but it is displayed in local time, and it's not clear whether that's in the user's timezone, or the timezone of the activity. It seems cleaner to instead assume that the tweet has been posted as soon as the activity is finished and store that time as UTC (this assumption may of course not be true, but the results seem realistic).<br />
<h3>
Show me the Code!</h3>
The code's not in a bad shape, but I would like to tidy it up a little before releasing it into the world. I'm busy with other things at the moment, but if I get much interest I can go ahead and do that... Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com1tag:blogger.com,1999:blog-848045343259901678.post-28430307757446600482013-07-17T11:39:00.001-07:002013-07-17T11:53:04.781-07:00Sipping from the twitter streamI’ve been playing around with connecting to twitter’s streaming API, and displaying a live stream of tweets returned.<br />
To do this, I was originally using <a href="http://dotnetopenauth.net/">DotNetOpenAuth</a>, along with the <a href="http://msdn.microsoft.com/en-us/library/system.net.http.httpclient.aspx">HttpClient</a>, which worked fine for the sample stream, but would return authentication errors for the filtered stream. I looked at the HTML message in fiddler2, and the oauth parameters weren’t ordered, which twitter requires. Instead, I’m using <a href="https://github.com/WebAPIDoodle/TwitterDoodle">TwitterDoodle</a>, which uses HttpClient.<br />
The C#5/.NET4.5 async/await code is quite elegant – it’s not CPU intensive so it’s fine running on the UI thread, without blocking. My first instinct prior to C#5 would have been to use RX for this, but now if I’m doing something simple I’d stick to async/await, only using RX if doing something more complex like buffering or batching.<br />
<br />
<pre style="background: #1e1e1e; color: gainsboro; font-family: Consolas; font-size: 13;"> <span style="color: #569cd6;">private</span> <span style="color: #569cd6;">async</span> <span style="color: #569cd6;">void</span> <span style="color: white;">ProcessTweets</span>()
{
<span style="color: #569cd6;">using</span> (<span style="color: #569cd6;">var</span> <span style="color: white;">t</span> <span style="color: #b4b4b4;">=</span> <span style="color: #569cd6;">new</span> <span style="color: #4ec9b0;">TwitterConnector</span>())
{
<span style="color: #569cd6;">var</span> <span style="color: white;">response</span> <span style="color: #b4b4b4;">=</span> <span style="color: #569cd6;">await</span> <span style="color: white;">t</span><span style="color: #b4b4b4;">.</span><span style="color: white;">GetSampleFirehoseConnection</span>();
<span style="color: #569cd6;">var</span> <span style="color: white;">res</span> <span style="color: #b4b4b4;">=</span> <span style="color: #569cd6;">await</span> <span style="color: white;">response</span><span style="color: #b4b4b4;">.</span><span style="color: white;">Content</span><span style="color: #b4b4b4;">.</span><span style="color: white;">ReadAsStreamAsync</span>();
<span style="color: #569cd6;">using</span> (<span style="color: #569cd6;">var</span> <span style="color: white;">streamReader</span> <span style="color: #b4b4b4;">=</span> <span style="color: #569cd6;">new</span> <span style="color: #4ec9b0;">StreamReader</span>(<span style="color: white;">res</span>, <span style="color: #4ec9b0;">Encoding</span><span style="color: #b4b4b4;">.</span><span style="color: white;">UTF8</span>))
{
<span style="color: #608b4e;">// streamReader.EndOfStream can block, so instead check for null</span>
<span style="color: #569cd6;">while</span> (<span style="color: #b4b4b4;">!</span><span style="color: white;">cts</span><span style="color: #b4b4b4;">.</span><span style="color: white;">IsCancellationRequested</span>)
{
<span style="color: #569cd6;">var</span> <span style="color: white;">r</span> <span style="color: #b4b4b4;">=</span> <span style="color: #569cd6;">await</span> <span style="color: white;">streamReader</span><span style="color: #b4b4b4;">.</span><span style="color: white;">ReadLineAsync</span>();
<span style="color: #569cd6;">if</span> (<span style="color: white;">r</span> <span style="color: #b4b4b4;">==</span> <span style="color: #569cd6;">null</span>) { <span style="color: #569cd6;">return</span>; }
<span style="color: white;">ProcessTweetText</span>(<span style="color: white;">r</span>);
}
}
}
}
<span style="color: #569cd6;">private</span> <span style="color: #569cd6;">void</span> <span style="color: white;">ProcessTweetText</span>(<span style="color: #569cd6;">string</span> <span style="color: white;">r</span>)
{
<span style="color: #569cd6;">if</span> (<span style="color: #b4b4b4;">!</span><span style="color: #569cd6;">string</span><span style="color: #b4b4b4;">.</span><span style="color: white;">IsNullOrEmpty</span>(<span style="color: white;">r</span>))
{
<span style="color: #569cd6;">var</span> <span style="color: white;">tweetJToken</span> <span style="color: #b4b4b4;">=</span> <span style="color: #4ec9b0;">JsonConvert</span><span style="color: #b4b4b4;">.</span><span style="color: white;">DeserializeObject</span><span style="color: #b4b4b4;"><</span><span style="color: #569cd6;">dynamic</span><span style="color: #b4b4b4;">></span>(<span style="color: white;">r</span>);
<span style="color: #569cd6;">var</span> <span style="color: white;">tweetObj</span> <span style="color: #b4b4b4;">=</span> <span style="color: white;">tweetJToken</span>[<span style="color: #d69d85;">"text"</span>];
<span style="color: #569cd6;">if</span> (<span style="color: white;">tweetObj</span> <span style="color: #b4b4b4;">!=</span> <span style="color: #569cd6;">null</span>)
{
<span style="color: #569cd6;">var</span> <span style="color: white;">tweetText</span> <span style="color: #b4b4b4;">=</span> <span style="color: white;">tweetObj</span><span style="color: #b4b4b4;">.</span><span style="color: white;">ToString</span>();
<span style="color: white;">viewModel</span><span style="color: #b4b4b4;">.</span><span style="color: white;">Items</span><span style="color: #b4b4b4;">.</span><span style="color: white;">Add</span>(<span style="color: white;">tweetText</span>);
}
}
}
</pre>
<br />The equivalent F# async code obviously looks quite similar, with the added goodness of Type Providers. Time dependent, I am planning to do some more analysis of tweets which would be a good fit for F#.<br />
<br />
<pre style="background: #1e1e1e; color: gainsboro; font-family: Consolas; font-size: 13;"><span style="color: #569cd6;">type</span> <span style="color: white;">tweetProvider</span> <span style="color: #b4b4b4;">=</span> <span style="color: white;">JsonProvider</span><span style="color: #b4b4b4;"><</span><span style="color: #d69d85;">"SampleTweet.json"</span>, <span style="color: white;">SampleList</span><span style="color: #b4b4b4;">=</span><span style="color: #569cd6;">true</span><span style="color: #b4b4b4;">></span>
<span style="color: #569cd6;">type</span> <span style="color: white;">MainWindowViewModel</span>() <span style="color: #b4b4b4;">=</span>
<span style="color: #569cd6;">inherit</span> <span style="color: white;">ViewModelBase</span>()
<span style="color: #569cd6;">let</span> <span style="color: white;">items</span> <span style="color: #b4b4b4;">=</span> <span style="color: #569cd6;">new</span> <span style="color: white;">ObservableCollection</span><span style="color: #b4b4b4;"><</span><span style="color: white;">string</span><span style="color: #b4b4b4;">></span>()
<span style="color: #569cd6;">member</span> <span style="color: white;">x</span><span style="color: #b4b4b4;">.</span><span style="color: white;">Items</span>
<span style="color: #569cd6;">with</span> <span style="color: white;">get</span> () <span style="color: #b4b4b4;">=</span> <span style="color: white;">items</span>
<span style="color: #569cd6;">member</span> <span style="color: white;">x</span><span style="color: #b4b4b4;">.</span><span style="color: white;">ProcessTweet</span> <span style="color: white;">tweet</span> <span style="color: #b4b4b4;">=</span>
<span style="color: #569cd6;">let</span> <span style="color: white;">tweetParsed</span> <span style="color: #b4b4b4;">=</span> <span style="color: white;">tweetProvider</span><span style="color: #b4b4b4;">.</span><span style="color: white;">Parse</span>(<span style="color: white;">tweet</span>)
<span style="color: #569cd6;">match</span> <span style="color: white;">tweetParsed</span><span style="color: #b4b4b4;">.</span><span style="color: white;">Text</span> <span style="color: #569cd6;">with</span>
| <span style="color: white;">Some</span>(<span style="color: white;">v</span>) <span style="color: #569cd6;">-></span> <span style="color: white;">x</span><span style="color: #b4b4b4;">.</span><span style="color: white;">Items</span><span style="color: #b4b4b4;">.</span><span style="color: white;">Add</span> <span style="color: white;">v</span>
| <span style="color: white;">None</span> <span style="color: #569cd6;">-></span> ()
<span style="color: #569cd6;">member</span> <span style="color: white;">x</span><span style="color: #b4b4b4;">.</span><span style="color: white;">ProcessTweets</span> <span style="color: #b4b4b4;">=</span>
<span style="color: #569cd6;">let</span> <span style="color: white;">a</span> <span style="color: #b4b4b4;">=</span> <span style="color: #569cd6;">async</span> {
<span style="color: #569cd6;">use</span> <span style="color: white;">t</span> <span style="color: #b4b4b4;">=</span> <span style="color: #569cd6;">new</span> <span style="color: white;">TwitterConnector</span>()
<span style="color: #569cd6;">let!</span> <span style="color: white;">response</span> <span style="color: #b4b4b4;">=</span> <span style="color: white;">t</span><span style="color: #b4b4b4;">.</span><span style="color: white;">GetSampleFirehoseConnection</span>() <span style="color: #b4b4b4;">|></span> <span style="color: white;">Async</span><span style="color: #b4b4b4;">.</span><span style="color: white;">AwaitTask</span>
<span style="color: #569cd6;">let!</span> <span style="color: white;">result</span> <span style="color: #b4b4b4;">=</span> <span style="color: white;">response</span><span style="color: #b4b4b4;">.</span><span style="color: white;">Content</span><span style="color: #b4b4b4;">.</span><span style="color: white;">ReadAsStreamAsync</span>() <span style="color: #b4b4b4;">|></span> <span style="color: white;">Async</span><span style="color: #b4b4b4;">.</span><span style="color: white;">AwaitTask</span>
<span style="color: #569cd6;">use</span> <span style="color: white;">streamReader</span> <span style="color: #b4b4b4;">=</span> <span style="color: #569cd6;">new</span> <span style="color: white;">StreamReader</span>(<span style="color: white;">result</span>, <span style="color: white;">Encoding</span><span style="color: #b4b4b4;">.</span><span style="color: white;">UTF8</span>)
<span style="color: #569cd6;">let</span> <span style="color: #569cd6;">rec</span> <span style="color: white;">processTweetsAsync</span> (<span style="color: white;">s</span><span style="color: #b4b4b4;">:</span><span style="color: white;">StreamReader</span>) <span style="color: #b4b4b4;">=</span>
<span style="color: #569cd6;">async</span> {
<span style="color: #569cd6;">let!</span> <span style="color: white;">r</span> <span style="color: #b4b4b4;">=</span> <span style="color: white;">s</span><span style="color: #b4b4b4;">.</span><span style="color: white;">ReadLineAsync</span>() <span style="color: #b4b4b4;">|></span> <span style="color: white;">Async</span><span style="color: #b4b4b4;">.</span><span style="color: white;">AwaitTask</span>
<span style="color: #608b4e;">// streamReader.EndOfStream can block, so instead check for null</span>
<span style="color: #569cd6;">if</span> <span style="color: white;">r</span> <span style="color: #b4b4b4;"><></span> <span style="color: #569cd6;">null</span> <span style="color: #569cd6;">then</span>
<span style="color: white;">x</span><span style="color: #b4b4b4;">.</span><span style="color: white;">ProcessTweet</span> <span style="color: white;">r</span>
<span style="color: #569cd6;">return!</span> <span style="color: white;">processTweetsAsync</span> <span style="color: white;">s</span>
}
<span style="color: #569cd6;">do!</span> <span style="color: white;">processTweetsAsync</span> <span style="color: white;">streamReader</span>
}
<span style="color: white;">a</span> <span style="color: #b4b4b4;">|></span> <span style="color: white;">Async</span><span style="color: #b4b4b4;">.</span><span style="color: white;">StartImmediate</span>
<span style="color: #569cd6;">member</span> <span style="color: white;">x</span><span style="color: #b4b4b4;">.</span><span style="color: white;">GoCommand</span> <span style="color: #b4b4b4;">=</span>
<span style="color: #569cd6;">new</span> <span style="color: white;">RelayCommand</span> ((<span style="color: #569cd6;">fun</span> <span style="color: white;">canExecute</span> <span style="color: #569cd6;">-></span> <span style="color: #569cd6;">true</span>),
(<span style="color: #569cd6;">fun</span> <span style="color: white;">action</span> <span style="color: #569cd6;">-></span> <span style="color: white;">x</span><span style="color: #b4b4b4;">.</span><span style="color: white;">ProcessTweets</span>))</pre>
Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com2tag:blogger.com,1999:blog-848045343259901678.post-32449852621085443592013-07-01T14:13:00.001-07:002013-07-17T11:47:35.863-07:00Picking with DirectX 11 and Bullet Physics<p>I updated the falling cubes demo to separate out the rendering and physics into separate WinRT components - see the WinRT subfolder under <a href="http://github.com/taumuon/Taumuon.Game">http://github.com/taumuon/Taumuon.Game</a> . The performance of this wasn't noticeably different when consumed by a WP8 DirectX application, but I then switched over to a "DirectX with XAML application", so that the application was hosted via C#, and consumed the WinRT components via a DrawingSurfaceBackgroundGrid, and performance fell off a cliff. <p>I created the C# host as I intended to put as much game logic as I could into an F# portable library, and the only way to do this is to have a C# hosting application assembly as WP8 apps, unlike Windows Store apps, don't allow WinRT(P) components to be created in anything other than C++. <p>Instead, of fighting the performance issues, I decided that it would be better to implement the game logic purely in C++ - see the WinRT subfolder under <a href="http://github.com/taumuon/Taumuon.Game">http://github.com/taumuon/Taumuon.Game</a> . This replaces most uses of the C++/CX extensions with standard C++ types, and implements picking, loosely based off <a href="http://halogenica.net/ray-casting-and-picking-using-bullet-physics">http://halogenica.net/ray-casting-and-picking-using-bullet-physics</a> <p>It gives a childish sense of satisfaction to do something as simple as destroy a wall of bricks so that the wall collapses. <p>Once I get time, the next step is to make the wall collapse more realistically by adding spring constraints between adjacent blocks, and to remove those springs once they are stretched beyond some elastic limit.</p> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com2tag:blogger.com,1999:blog-848045343259901678.post-22997750274446330962013-05-18T14:10:00.001-07:002013-07-17T11:47:35.859-07:00Bullet Physics demo DirectX C++ app on Windows Phone 8<p>I’ve finally gotten around to updating my original Windows Store Bullet Physics demo (<a href="http://taumuon-jabuka.blogspot.co.uk/2012/04/bullet-physics-and-directx-11-on.html">http://taumuon-jabuka.blogspot.co.uk/2012/04/bullet-physics-and-directx-11-on.html</a>) to build for the Windows 8 final release. <p>While I was at it, I also got it working for Windows Phone 8 – code is on github <a href="https://github.com/taumuon/WP8-Bullet">https://github.com/taumuon/WP8-Bullet</a>. There weren’t many changes. In building Bullet for ARM, I got around lots of parameter alignment issues by defining BT_USE_DOUBLE_PRECISION. This was the quickest thing to do, and may have performance implications but the performance running on my Nokia Lumia 920 seems encouraging. It wouldn’t have been a much bigger change to instead see if ARM is defined. <p> </p> <div style="padding-bottom: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; float: none; padding-top: 0px" id="scid:5737277B-5D6D-4f48-ABFC-DD9C333F4C5D:7a611e93-d618-493f-af07-5c19d84f66ba" class="wlWriterEditableSmartContent"><div id="4304b3a5-dd13-4857-b6fc-2445f87edb96" style="margin: 0px; padding: 0px; display: inline;"><div><a href="http://www.youtube.com/watch?v=apjczngx_nQ" target="_new"><img src="http://lh4.ggpht.com/-M-CQjt28kwc/UZfuNHHwmWI/AAAAAAAAAsg/tGrc59qkWIM/video1a6379c2737e%25255B4%25255D.jpg?imgmax=800" style="border-style: none" galleryimg="no" onload="var downlevelDiv = document.getElementById('4304b3a5-dd13-4857-b6fc-2445f87edb96'); downlevelDiv.innerHTML = "<div><object width=\"448\" height=\"252\"><param name=\"movie\" value=\"http://www.youtube.com/v/apjczngx_nQ?hl=en&hd=1\"><\/param><embed src=\"http://www.youtube.com/v/apjczngx_nQ?hl=en&hd=1\" type=\"application/x-shockwave-flash\" width=\"448\" height=\"252\"><\/embed><\/object><\/div>";" alt=""></a></div></div></div> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-77169025361884950542013-03-12T13:47:00.001-07:002013-03-12T13:47:58.228-07:00ViewGene now has mapping<p>The newly-released version of my genealogy/family tree Windows 8 app now has Bing Maps integration, which allows the migration paths of all ancestors to be plotted.</p> <p>I wanted to see visually how my ancestors moved, after researching how my ancestors moved from villages in the 17th century into towns, and then larger towns.</p> <p>I’d love to see anyone else’s migration paths!</p> <p>The below is just a made up sample – I’ll share my map after sanitising the data.</p> <p><a href="http://lh4.ggpht.com/-ueB8PPp2o8k/UT-Ud2gNjxI/AAAAAAAAArg/UKEyaJ_bTxw/s1600-h/ViewGeneMapping%25255B3%25255D.jpg"><img title="ViewGeneMapping" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="ViewGeneMapping" src="http://lh4.ggpht.com/-Zp35ATvRLFs/UT-Ue7ZBMMI/AAAAAAAAAro/gWFMMZxpKfc/ViewGeneMapping_thumb%25255B1%25255D.jpg?imgmax=800" width="486" height="291"></a></p> <p>Try it in the <a href="http://apps.microsoft.com/windows/en-US/app/viewgene/601eb3b1-e238-40f5-96bc-9c8d8b52b78d">Windows Store</a>.</p> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-21324416468300345802013-02-20T13:27:00.001-08:002014-04-25T14:06:50.770-07:00Option pricing in F# using the Explicit Finite Difference method<p>It's been a little while since I've coded any F#, so I've done a little coding kata to polish off the rust.</p> <p>I've converted the explicit finite difference option pricing example from (<a href="http://www.amazon.co.uk/dp/0470319585/?tag=hydra0b-21&hvadid=9556637709&ref=asc_df_0470319585">Paul Wilmott Introduces Quantitative Finance</a>).</p> <p><a href="http://lh4.ggpht.com/-fMxobOV760U/USU_qG7uowI/AAAAAAAAAq4/qYLkL1peS58/s1600-h/Call%25255B3%25255D.png"><img title="Call" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="Call" src="http://lh3.ggpht.com/-AkFqbGLa8P0/USU_q0QqjbI/AAAAAAAAArA/qc7QWZ5T-jk/Call_thumb%25255B1%25255D.png?imgmax=800" width="381" height="244"></a></p> <p>I followed similar steps as those I performed in my previous <a href="http://taumuon-jabuka.blogspot.co.uk/2012/07/binomial-tree-option-pricing-in-f.html">blog post on the binomial tree option pricing</a>; I converted the VBA code into very similar looking imperative F# code, before refactoring out the loops into recursive calls.</p> <p>It was a useful exercise as converting the algorithm made me actually focus on the mechanics of how it worked, much more than simply translating VBA code into imperative F#. It’d be interesting to hear in the comments whether people find it easy to read.</p> <p>The code is available <a href="https://gist.github.com/taumuon/4999749">https://gist.github.com/taumuon/4999749</a>.</p> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-17095481559755593822013-02-06T14:27:00.001-08:002013-02-06T14:27:47.491-08:00ViewGene<p>This blog has been quiet for a while, but I've been busy working on a Windows Store genealogy application, <a href="http://apps.microsoft.com/windows/app/viewgene/601eb3b1-e238-40f5-96bc-9c8d8b52b78d">ViewGene</a>. <p>This is quite a niche application, but it was written to cope with a couple of use cases which are missing from other family tree websites and applications – it's useful to have it open in conjunction with your favourite family tree software or website. <p>The first problem is that many of the family tree websites, when showing the whole tree (pedigree chart) of ancestors, compress the tree to make the best use of screen real estate. This is fine to save paper, but it does make it very difficult, when navigating the tree, to know what the generation is of any ancestor on the screen. <p><a href="http://lh4.ggpht.com/-D0JHl-QY4WM/URLY29o2sOI/AAAAAAAAAp8/XbFSUPfT5Uo/s1600-h/ViewGenePedigree%25255B3%25255D.jpg"><img title="ViewGenePedigree" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="ViewGenePedigree" src="http://lh4.ggpht.com/-a0usxtWSL0s/URLY3fA8YkI/AAAAAAAAAqE/i7qTRTcynoM/ViewGenePedigree_thumb%25255B1%25255D.jpg?imgmax=800" width="486" height="291"></a> <p>ViewGene ensures that ancestors of any generation all appear at the same vertical level – this makes it easy to see which branches of the tree need more investigation. This view wouldn't necessarily work well when printed out, but touch-based pan and zoom do mean that it's easy to zoom out to see the 'shape' of the tree, and to zoom in to see the individual details, meaning that it works for interactively investigating the data. <p>The second use-case is to validate and see the lifelines of the ancestors. Many of the family tree websites allow free-form entry of dates. This means that it is easy to mistype and enter a date of the wrong century, or to enter semantically incorrect dates such as a child born before the father. The Timeline Fan chart allows for some of these problems to be easily seen. The chart also allows the user to see, for example, which ancestors were alive on a given census year. <p><a href="http://lh6.ggpht.com/-riMWwZ-6ceI/URLY4CI_kZI/AAAAAAAAAqM/EQn9l973yW8/s1600-h/ViewGeneTimelineFan%25255B3%25255D.jpg"><img title="ViewGeneTimelineFan" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="ViewGeneTimelineFan" src="http://lh3.ggpht.com/-QfMQ6Ah94iI/URLY4s5SKdI/AAAAAAAAAqU/jL0VAFgeZiE/ViewGeneTimelineFan_thumb%25255B1%25255D.jpg?imgmax=800" width="495" height="297"></a> <p>The current release includes very limited validation, to check that dates match allowable GEDCOM formats, but does not check for semantically incorrect dates. Also, there is very basic date inference for missing dates. If I devote any more time to the application, I'll want to improve both of these points – it should be possible to validate that children were conceived whilst both parents were alive, and that siblings can't have been born in the same gestation period etc. <h2>Windows Store application observations</h2> <p>The chart functionality is quite generic, and would work well on any touch-screen platform (such as iOS), much of the development effort was spent in building an application following Windows Store idioms. <p>The first was to ensure that the application works when snapped (though this could do with further improvement). The second was to use semantic zoom, which definitely makes it easier to navigate through a large number of individuals in the GEDCOM file). Adding a very large number of individuals to the GridView results in Invalid Quota exceptions being thrown. Even if this weren't the case, there would be a usability issue in navigating through so many individuals. To fix this, if there are a large number of individuals in the GEDCOM file the application adds a further level of navigation through the first letter of the surname. <p>It would be quite easy to add images to the application (such as pictures of ancestors, or scans or pdfs of documentation), but it is quite tricky in a Windows Store application – the GEDCOM file contains paths to images, but it is not possible from a Windows Store application to open files other than via the open file dialog. It may be possible to work around that by instead presenting an open folder dialog (which would allow access to all files and sub-folders within that folder) and then allowing the user to choose the GEDCOM file from a list, but that definitely feels clunky. <p>The Pedigree Chart is implemented as an ItemsControl, populated with either Boxes or Lines, with an ItemsTemplateSelector to distinguish between them. It works well, and there are no obvious performance issues, but the ItemsControl does crash if too many items are added, so the number of ancestors are limited to 1000. <p>The timeline fan is implemented as C++/CX WinRT component, deriving from SurfaceImageSource, with all drawing being performed in Direct2D. If I get time to invest in this, I want to switch out to a VirtualSurfaceImageSource, so that the amount of information presented is customised to the zoom level on the screen. <h2>Non Windows Store issues</h2> <p>There are a few issues not related to a Windows Store application. <p>The application currently does not have editing – the GEDCOM format is too flaky to be a reliable interchange format; it does not round-trip without the possibility of corrupting the user's information. Ideally, the application would allow connecting to a website to obtain the user's family tree information, but ancestry.com does not provide a user-accessible API. Geni.com does, and if I get time I intend to investigate that. <p>It is also trickier than it needs to be to implement mapping – it looks fairly easy to integrate with Bing maps, but the problem is that location data in the GEDCOM file doesn't necessarily match up with the required data for plotting points on the map. It is possible to ask the data to enter that information, but then the application should maintain it. This isn't too much of a problem, but it should be maintained per GEDCOM file, and to allow the user to re-import updated GEDCOM files would rely on GEDCOM merging. <p>The GEDCOM parser is written in C#, mostly as a GoF state machine, with some functional elements. When I originally started coding this up, I intended to start by creating a clunky procedural looking parser (which maintains the state with many flags), then show how an OO GoF pattern was nicer, then show how a F# implementation is nicer still. If I get time I still intend to create a F# version of the parser. <p>Speaking of F#, the layout logic for the timeline fan is implemented in a functional way, but would be nicer in F# - the problem is that it then needs to communicate with the WinRT component to do the drawing. F# can't directly interop with WinRT, so it would be necessary to create a C# wrapper or consumer of the F# library just for this – and it seems overkill. <p>Overall, developing a Windows Store app is fairly easy if you have Silverlight or WPF skills. There are quite a number of features I want to add to the application, but how much time I'm going to dedicate to it depends on feedback or download numbers of the current version of the app. Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-51001210660996644282012-07-16T13:11:00.001-07:002014-04-25T14:06:50.776-07:00Binomial Tree Option Pricing in F#<p>This post will look at implementing the binomial tree algorithm in an imperative style, and refactoring it to be more functional. <p>The <a href="http://en.wikipedia.org/wiki/Binomial_options_pricing_model">wikipedia page</a> on the binomial options pricing model provides pseudo-code for the pricing of an American Put option. <p>That code can be implemented almost identically in F#. <p><a href="http://lh3.ggpht.com/-Dw7ZWzDFIaI/UAR1UH2mUPI/AAAAAAAAAbI/dXNzlDefbX0/s1600-h/clip_image002%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image002" border="0" alt="clip_image002" src="http://lh4.ggpht.com/-MLrtS2NgeXI/UAR1VVV2bZI/AAAAAAAAAbQ/mgv4jFdcgN0/clip_image002_thumb%25255B1%25255D.png?imgmax=800" width="495" height="438"></a> <p>and is called as follows: <p><a href="http://lh6.ggpht.com/-OS6ak1O-JEs/UAR1Wss7VMI/AAAAAAAAAbY/B_pLGIXtFJ0/s1600-h/clip_image004%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image004" border="0" alt="clip_image004" src="http://lh4.ggpht.com/-2sTBrjTiA4A/UAR1XTSyLPI/AAAAAAAAAbg/bekOkC2bnJs/clip_image004_thumb%25255B1%25255D.png?imgmax=800" width="370" height="278"></a> <p>Functional programmers have probably died a little on the inside on seeing that code, but one of the good thing about F# being multi-paradigm is that it's easy to take code of this form and implement it, and test it for correctness before refactoring. <p>I did actually look at whether the results of the code were as expected at this point, but I'll cover this later in this blog post. <p>The first thing to do is to change the initial values V to be created more functionally: <p><a href="http://lh6.ggpht.com/-CBPZue88aCQ/UAR1YJWhd8I/AAAAAAAAAbk/xDxb2B1-qKk/s1600-h/clip_image006%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image006" border="0" alt="clip_image006" src="http://lh6.ggpht.com/-z50wl0dynpo/UAR1Yl_fvaI/AAAAAAAAAbw/I8SsQxA6_N4/clip_image006_thumb%25255B1%25255D.png?imgmax=800" width="341" height="84"></a> <p>Now, the main algorithm consists of two parts, an outer loop where each 'level' of the tree is calculated stepping up towards the root, using the results of the previous level’s nodes (the nodes nearest to the leaves). The inner loop iterates over all of the nodes at a given level calculating the option's value at that node. <p>First off, the outer loop can be replaced with a recursive method: <p><a href="http://lh4.ggpht.com/-rqZJBF9m9J0/UAR1ZthZrsI/AAAAAAAAAb0/ggL_2XPVCdE/s1600-h/clip_image008%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image008" border="0" alt="clip_image008" src="http://lh5.ggpht.com/-a2Y3s9CAa2U/UAR1aTQLpgI/AAAAAAAAAcA/6YHNcZDXVuA/clip_image008_thumb%25255B1%25255D.png?imgmax=800" width="428" height="174"></a> <p>The next change is quite minor – it's for each pass of the inner loop to create a new array, instead of mutating the vi array: <p><a href="http://lh5.ggpht.com/-GtYzpO-I2Q8/UAR1bPcArVI/AAAAAAAAAcE/9QgTh7NRX-Y/s1600-h/clip_image010%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image010" border="0" alt="clip_image010" src="http://lh3.ggpht.com/-rNSxG5yHMa4/UAR1cFICpJI/AAAAAAAAAcQ/sx-3PHP-gJg/clip_image010_thumb%25255B1%25255D.png?imgmax=800" width="432" height="189"></a> <p>This change now means that there's no need for any mutable state, so the initial values can be created as a sequence instead of an array, and the inner loop can be written in a functional style: <p><a href="http://lh3.ggpht.com/-E2xNhvp8lK0/UAR1dNRKs9I/AAAAAAAAAcU/1AXGGhEW8ho/s1600-h/clip_image012%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image012" border="0" alt="clip_image012" src="http://lh4.ggpht.com/-tANQo_ldCbk/UAR1dnP5KgI/AAAAAAAAAcc/LYpn8tMI-gs/clip_image012_thumb%25255B1%25255D.png?imgmax=800" width="435" height="211"></a> <p>I find this code style is clearer to understand than following the array mutations to see what each part of the code is doing and how those parts interact. <p>If I was implementing this in C# I'd probably change the variable names to be more descriptive (e.g. strikePrice, riskFreeRate etc) but as one of the key advantages of F# is its succinctness it makes more sense to leave the well-known symbols in. Anyone implementing the algorithm with knowledge of the domain would know what the symbols mean without the more verbose names. <h4>Validating the results</h4> <p>The calculated option value with the requested parameters is 4.486090. Running the EqutyOptions example in the .NET port of QuantLib, QLNet (http://sourceforge.net/projects/qlnet), with the same parameters gives the following results: <p><a href="http://lh5.ggpht.com/-CbA39Kajj6U/UAR1emLiVlI/AAAAAAAAAco/EFcGl1P5fis/s1600-h/clip_image014%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image014" border="0" alt="clip_image014" src="http://lh4.ggpht.com/-BD5NXIcXLFw/UAR1fiFFwmI/AAAAAAAAAcw/zDdXwmuUIKk/clip_image014_thumb%25255B1%25255D.png?imgmax=800" width="421" height="246"></a> <p>The figures are in the same ballpark, but the difference is curious. <p><a href="http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter45.html">Chapter 45</a> of GPU GEMS 2 creates the initial values in this way: <p><a href="http://lh4.ggpht.com/-u2K8nD2ggSQ/UAR1g3fQOfI/AAAAAAAAAc0/z_QAhDy78Ps/s1600-h/clip_image016%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image016" border="0" alt="clip_image016" src="http://lh4.ggpht.com/-weyg9VNR04c/UAR1hojZAUI/AAAAAAAAAdA/XnvMjuvsLuw/clip_image016_thumb%25255B1%25255D.png?imgmax=800" width="363" height="97"></a> <p>This gives a result of 4.486375 which still doesn't match QLNet, but is nearer. <p>The result of running the Cox version of Bubo's code from here <a href="http://cs.hubfs.net/topic/None/59053">http://cs.hubfs.net/topic/None/59053</a> also equals 4.486375, which matches the results of using this initialisation function. As an aside, I like how that examples shows how F# partial function application allows the variations on the algorithm to be composed, and much more succinctly than using OO. <p>Bubo’s Tian model does match QuantLib's result exactly. I'll have to dig a bit deeper to figure out which <a href="http://www.goddardconsulting.ca/option-pricing-binomial-alts.html">variations</a> on the binomial tree model these different examples are using. <p>The code is available here: <a title="https://github.com/taumuon/FSharp-Binomial-Tree" href="https://github.com/taumuon/FSharp-Binomial-Tree">https://github.com/taumuon/FSharp-Binomial-Tree</a></p> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-82518666723862814852012-07-04T12:24:00.003-07:002013-05-18T14:07:40.329-07:00Metro Bullet source codeI've updated the demo that shows the Bullet Physics Engine in a DirectX 11 Metro application to build in the Release Preview drop of Visual Studio (original blog post: <a href="http://taumuon-jabuka.blogspot.co.uk/2012/04/bullet-physics-and-directx-11-on.html">http://taumuon-jabuka.blogspot.co.uk/2012/04/bullet-physics-and-directx-11-on.html</a> ).<br />
<br />
The code needs some tweaks - I need to change the manifest, and change some stuff over to Platform::Agile, but it builds and runs so I thought I'd get it out there.<br />
<br />
The code is available at: <a href="https://github.com/taumuon/Metro-Bullet">https://github.com/taumuon/Metro-Bullet</a>Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-2411523090964334182012-06-27T12:27:00.001-07:002012-06-27T12:35:14.637-07:00More playing with monads<p>This blog post will look at using the continuation monad to sum the nodes in a tree, and then look at a case where a more complex monad is needed. <p>The book <a href="http://manning.com/petricek/">Real World Functional Programming</a> gave an example of an unbalanced tree, which was deeply nested such that recursing over the tree to sum the values would blow the stack: <p><a href="http://lh5.ggpht.com/-LYI0bwerbvI/T-tenBPVKXI/AAAAAAAAAZE/5CpFQec1LcI/s1600-h/clip_image001%25255B6%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image001" border="0" alt="clip_image001" src="http://lh4.ggpht.com/--muUzzX-Mp4/T-tenefzONI/AAAAAAAAAZM/Ct38oucq8V4/clip_image001_thumb%25255B3%25255D.png?imgmax=800" width="358" height="157"></a> <p><a href="http://lh6.ggpht.com/-DAgzTspGRO0/T-ten2ZH1QI/AAAAAAAAAZU/3icBKMJ_c_I/s1600-h/clip_image002%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image002" border="0" alt="clip_image002" src="http://lh4.ggpht.com/-ewnC5-Q-5LY/T-teoCks9GI/AAAAAAAAAZc/cQjthyNMid4/clip_image002_thumb%25255B1%25255D.png?imgmax=800" width="480" height="184"></a> <p>The solution provided was to sum the tree using continuation passing: <p><a href="http://lh6.ggpht.com/-4VEtw_HQUz8/T-teoQNVZMI/AAAAAAAAAZk/o7RHFv4d0kw/s1600-h/clip_image003%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image003" border="0" alt="clip_image003" src="http://lh4.ggpht.com/-muXBp1JHYtA/T-teo-p1ilI/AAAAAAAAAZs/KHeeNvv5NSU/clip_image003_thumb%25255B1%25255D.png?imgmax=800" width="426" height="125"></a> <p>This is now able to sum the tree without crashing, but the code doesn’t flow as well as in the original sumTree method. However, monads can come to the rescue again, in this case the continuation monad: <p><a href="http://lh6.ggpht.com/-0kbGfEbOD9k/T-tepb6FxoI/AAAAAAAAAZ0/wg0zsVesObI/s1600-h/clip_image004%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image004" border="0" alt="clip_image004" src="http://lh4.ggpht.com/-uSlNChmGQRs/T-tep8zZMDI/AAAAAAAAAZ8/8F2kD6k0xe8/clip_image004_thumb%25255B1%25255D.png?imgmax=800" width="443" height="245"></a> <p>The code flow now is easier to understand. The continuation passing style and continuation monad turns the code flow inside out, which isn’t a big deal, but means that the func to perform on the result needs to be passed in: <p><a href="http://lh5.ggpht.com/-Zho0ZevRyAI/T-teqP5ldRI/AAAAAAAAAaE/Rj_vk5gWhQs/s1600-h/clip_image005%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image005" border="0" alt="clip_image005" src="http://lh5.ggpht.com/-NkyE4DDgmSY/T-teqcYwGkI/AAAAAAAAAaM/v0QIEbF1idI/clip_image005_thumb%25255B1%25255D.png?imgmax=800" width="430" height="55"></a> <h3>Parallel sum</h3> <p>This is all well and good, but to exploit multicore machines, I want to take advantage of the parallelism in the tree summation algorithm, by summing the different branches of the nodes using tasks. A naïve implementation to do this may be: <p><a href="http://lh3.ggpht.com/-Ba_dxhgx6mI/T-teqrK9OFI/AAAAAAAAAaU/U6_GWfXjmcc/s1600-h/clip_image006%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image006" border="0" alt="clip_image006" src="http://lh4.ggpht.com/-9xvX77hS7nk/T-terGMYlRI/AAAAAAAAAac/uJNs0IN4AFo/clip_image006_thumb%25255B1%25255D.png?imgmax=800" width="428" height="151"></a> <p>The problem with this is that it runs slower than the sequential case for a large balanced tree –if there are only 4 cores in the system it’s counter-productive to create thousands of tasks. The current depth of the tree should be passed around, and tasks spawned until traversal reaches that depth: <p><a href="http://lh3.ggpht.com/-bkGhhjYqTss/T-terRpSmoI/AAAAAAAAAak/W5kY2YlEytM/s1600-h/clip_image007%25255B5%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image007" border="0" alt="clip_image007" src="http://lh6.ggpht.com/-zio4ycFLFqI/T-ter9xf8iI/AAAAAAAAAas/Z-y_YkVjQrU/clip_image007_thumb%25255B2%25255D.png?imgmax=800" width="494" height="203"></a> <p>This is now faster for a large balanced tree, but for the deeply nested unbalanced tree it throws a StackOverflowException. Note that sumTreeParallel coped with arbitrarily nested trees, as the work to be done no longer kept being pushed onto the stack. <p>More state is being passed around (the current depth), which is external to the main <i>business logic</i> – it does feel like the state monad could be used here. First though, this is fixed up so that it has the efficiency of the parallel summation, but can cope with arbitrarily deeply nested trees: <p><a href="http://lh3.ggpht.com/-KjXTMjL1fck/T-tesDbDZxI/AAAAAAAAAa0/4VUZBspD1f8/s1600-h/clip_image008%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image008" border="0" alt="clip_image008" src="http://lh6.ggpht.com/-V4ivCVY-sqE/T-tes83jgCI/AAAAAAAAAa8/1_ChKkcEI0M/clip_image008_thumb%25255B1%25255D.png?imgmax=800" width="374" height="390"></a> <p>Urghhhh. The code is definitely more convoluted now – I played with using the state monad to pass the depth around, but it’s not tail recursive. I thought that it might be possible to make use of the continuation monad here, but the problem is, is that it really needs aspects of both the state and continuation monads. I’m not sure how monad combinations work in F#, but I thought I’d throw this out here to see if anyone shows me the light. The post <a href="http://blog.sigfpe.com/2008/12/mother-of-all-monads.html">The Mother of All Monads</a> does say that all other monads could be implemented in terms of the continuation monad, but I couldn’t see an example of anyone implementing the state monad in terms of the continuation monad. Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com2tag:blogger.com,1999:blog-848045343259901678.post-36923665981309100232012-06-06T12:04:00.001-07:002012-06-06T12:22:51.028-07:00State Monad in C# and F#<p>My colleagues and I recently have been trying to gain a deeper understanding of monads, other than using LINQ and RX. Watching all the channel9 videos on monads, the concept on the surface seemed quite simple, but we were felt that we were somehow missing some key point. After reading various blog posts, the penny is finally dropping. <p>Some post I read somewhere (I can’t remember where) said that it’s easier to understand the individual monads before trying to understand the whole generalised concept. Mike Hadlow’s blog series (<a href="http://mikehadlow.blogspot.co.uk/2011/01/monads-in-c1-introduction.html">http://mikehadlow.blogspot.co.uk/2011/01/monads-in-c1-introduction.html</a>) really explain well the identity and maybe monads (which we already understood, which is just as well as we’re using the maybe monad in our codebase). Unfortunately he didn’t explain the state monad. <p>The only information we could find on the state monad in C# was Brian Beckman’s Channel 9 video: <a href="http://channel9.msdn.com/Shows/Going+Deep/Brian-Beckman-The-Zen-of-Expressing-State-The-State-Monad">http://channel9.msdn.com/Shows/Going+Deep/Brian-Beckman-The-Zen-of-Expressing-State-The-State-Monad</a> <p>We found Brian Beckman’s example overly complex, there’s quite a lot going on in that video (and the code needed de-obfuscating to read more like idiomatic C#), but it’s probably a good second or third example once the basics are understood. I wanted to look at a simpler example, and found this article: <a href="http://ertes.de/articles/monads.html#section-6">http://ertes.de/articles/monads.html#section-6</a> the explanation of the get and put methods show pretty clearly how the state monad works, and you should probably read and understand that section before carrying on here, as I’m going to jump straight on to show how the random number generator example looks in F# and C#. <h3>C# implementation</h3> <p>First off, here’s an OO random number generator: <p><a href="http://lh3.ggpht.com/-1tSRT62Qklo/T8-pLhURR3I/AAAAAAAAAUs/s7KQ-AUNKoA/s1600-h/clip_image001%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image001" border="0" alt="clip_image001" src="http://lh6.ggpht.com/-66QJJHkw-HI/T8-pMKqeVgI/AAAAAAAAAU0/F4tHTPBdmNU/clip_image001_thumb%25255B1%25255D.png?imgmax=800" width="469" height="275"></a> <p>It is easy to add three random numbers together using an instance of the generator. <p><a href="http://lh3.ggpht.com/-4uxf0O6BvqE/T8-pNzDEljI/AAAAAAAAAU8/9mk0GPU8EqU/s1600-h/clip_image002%25255B5%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image002" border="0" alt="clip_image002" src="http://lh6.ggpht.com/-mOurd5qhXGo/T8-pOrqYSTI/AAAAAAAAAVE/lKifwCy7eQU/clip_image002_thumb%25255B2%25255D.png?imgmax=800" width="367" height="204"></a> <p>We can make NextShort purely functional, so that calling it has no side-effects, by passing the current state in, and returning the new state back out, along with the result: <p><a href="http://lh4.ggpht.com/-clD6ZFxdgKQ/T8-pPERsqTI/AAAAAAAAAVM/J5c2RLleHCY/s1600-h/clip_image003%25255B5%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image003" border="0" alt="clip_image003" src="http://lh6.ggpht.com/-y6-vpK7Ydjo/T8-pPr4gm2I/AAAAAAAAAVU/Elp-iAG1MJY/clip_image003_thumb%25255B2%25255D.png?imgmax=800" width="463" height="212"></a> <p>To add three numbers, the state has to be kept around, and passed to each of the calls in turn. This obfuscates the main business logic. <p><a href="http://lh6.ggpht.com/-dOvHaBh-pJY/T8-pSG1W73I/AAAAAAAAAVg/nJ9Mm6hQDK4/s1600-h/clip_image004%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image004" border="0" alt="clip_image004" src="http://lh5.ggpht.com/-48-YGWjH_ks/T8-pSmy73MI/AAAAAAAAAVo/AlIK2Uq85ik/clip_image004_thumb%25255B1%25255D.png?imgmax=800" width="441" height="184"></a> <p>Jumping straight to the punch line, using a state monad means that the boilerplate state management can be <i>hidden away</i> so that the algorithm reads as clearly as it did before: <p><a href="http://lh5.ggpht.com/-8EMc8m5c0_M/T8-pTdkIwUI/AAAAAAAAAVw/oGhEr7NV_Mg/s1600-h/clip_image005%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image005" border="0" alt="clip_image005" src="http://lh6.ggpht.com/-JXtSX2E32fc/T8-pT_NzgwI/AAAAAAAAAV4/4MD5BJqXx-g/clip_image005_thumb%25255B1%25255D.png?imgmax=800" width="440" height="207"></a> <p>This is a trivial example, but hopefully it makes the point. The rest of this blog post will show the implementation, along with the same in F#. <p>First off, we have a state class, which holds a Func that given a state, returns a tuple of the state and value. The F# example doesn’t bother with the State class, and I could have made the C# version simply use the Func everywhere, but it seemed clearer to me to have the wrapper object just to get a bit of context. <p><a href="http://lh4.ggpht.com/-x_mEnW2eBFI/T8-pUL-gAfI/AAAAAAAAAWA/RR2hmzUbF7A/s1600-h/clip_image006%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image006" border="0" alt="clip_image006" src="http://lh4.ggpht.com/-Hl3ep5wbcvw/T8-pUhQuLqI/AAAAAAAAAWI/oetW-UZ0TBk/clip_image006_thumb%25255B1%25255D.png?imgmax=800" width="440" height="104"></a> <p>I created a non-generic State class as somewhere convenient to put the Get and Set methods: <p><a href="http://lh3.ggpht.com/-BC-bxxTKtds/T8-pVCorUlI/AAAAAAAAAWQ/K4ZD-GG9s6o/s1600-h/clip_image007%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image007" border="0" alt="clip_image007" src="http://lh4.ggpht.com/-zkvclYJfgK4/T8-pVe6baXI/AAAAAAAAAWY/aZ4A5LGgbVM/clip_image007_thumb%25255B1%25255D.png?imgmax=800" width="459" height="347"></a> <p>The meat of the monad lives in extension methods in a StateEx class: <p><a href="http://lh6.ggpht.com/-p1gyOifEuNw/T8-pV7XrtUI/AAAAAAAAAWg/IIKI2d3n4HM/s1600-h/clip_image008%25255B6%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image008" border="0" alt="clip_image008" src="http://lh5.ggpht.com/-k_SpEDd4aGg/T8-pWYaEmcI/AAAAAAAAAWo/EUGak_ww8lc/clip_image008_thumb%25255B3%25255D.png?imgmax=800" width="498" height="329"></a> <p>and SelectMany is implemented in terms of Bind the same as in Mike Hadlow’s example for the Identity and Maybe monads: <p><a href="http://lh4.ggpht.com/-QOiRq-hgEGc/T8-pWy3R_5I/AAAAAAAAAWw/vw0--BmmdGs/s1600-h/clip_image009%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image009" border="0" alt="clip_image009" src="http://lh4.ggpht.com/-IxyPwZ2r21g/T8-pYAvEl2I/AAAAAAAAAW4/90Mpsec2A0w/clip_image009_thumb%25255B1%25255D.png?imgmax=800" width="453" height="195"></a> <p>The computation to compose can be expressed in the GetRandom method: <p><a href="http://lh5.ggpht.com/-o9IlAv-nTlg/T8-pZ3XW0MI/AAAAAAAAAXA/-VzQzinqvo4/s1600-h/clip_image010%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image010" border="0" alt="clip_image010" src="http://lh5.ggpht.com/-YD0dHBL2tP0/T8-paFdRlmI/AAAAAAAAAXI/chGT1Chtbf4/clip_image010_thumb%25255B1%25255D.png?imgmax=800" width="460" height="226"></a> <p>This returns an operation (a Func in the State container) that invokes NextShort with the old state, and returns a computation with the new state and value. <p>The resulting composed operation using LINQ is described above, but this is what it looks like desugared: <p><a href="http://lh5.ggpht.com/-oMoyfNSYsTY/T8-pan_PUQI/AAAAAAAAAXQ/5TfpIX-j7PY/s1600-h/clip_image011%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image011" border="0" alt="clip_image011" src="http://lh5.ggpht.com/-07d8WuObudo/T8-pbcAAb2I/AAAAAAAAAXY/oJzBGXewvNU/clip_image011_thumb%25255B1%25255D.png?imgmax=800" width="481" height="183"></a> <h3>F# implementation</h3> <p>The code for the random number generator and composed operation look similar to the C#: <p><a href="http://lh3.ggpht.com/-7DiT8W2k8zM/T8-pbwvE5II/AAAAAAAAAXg/NPpMvyirW2g/s1600-h/clip_image012%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image012" border="0" alt="clip_image012" src="http://lh6.ggpht.com/-u0j1NlqKass/T8-pcOd-GDI/AAAAAAAAAXo/ebPwvHQWfkY/clip_image012_thumb%25255B1%25255D.png?imgmax=800" width="456" height="228"></a> <p>This has the same <i>problem </i>of explicitly passing the state around. I can ruin the surprise again and show how using monads hides the explicit state passing: <p><a href="http://lh6.ggpht.com/-IplIrt8MOys/T8-pdwPPmiI/AAAAAAAAAXw/enJUDp0bOyU/s1600-h/clip_image013%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image013" border="0" alt="clip_image013" src="http://lh3.ggpht.com/-A5FnDAWq3Oo/T8-pemAxatI/AAAAAAAAAX4/8dRUnm8bHJc/clip_image013_thumb%25255B1%25255D.png?imgmax=800" width="295" height="262"></a> <p>This code makes use of F#’s support for monads via Computation Expressions. The code for the state builder is from here: <a href="http://www.navision-blog.de/2009/10/23/using-monads-in-fsharp-part-i-the-state-monad/">http://www.navision-blog.de/2009/10/23/using-monads-in-fsharp-part-i-the-state-monad/</a> <p><a href="http://lh5.ggpht.com/-Hr5ewBBOvoY/T8-pfdjYHxI/AAAAAAAAAYA/H8AqjWaChoQ/s1600-h/clip_image014%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image014" border="0" alt="clip_image014" src="http://lh6.ggpht.com/-MZC0nVD1A2Y/T8-pf9qXOAI/AAAAAAAAAYI/Ql97onuzdA8/clip_image014_thumb%25255B1%25255D.png?imgmax=800" width="312" height="380"></a> <p>The Haskell example also had the >> operator, but of course F# uses that as the forward composition operator J <p>Instead I’ve implemented: <p><a href="http://lh4.ggpht.com/-kDnwbmvtyG4/T8-pgMoIKEI/AAAAAAAAAYQ/VrtfWkgK6RA/s1600-h/clip_image015%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image015" border="0" alt="clip_image015" src="http://lh6.ggpht.com/-bkLrHaXZr10/T8-pgpjgQqI/AAAAAAAAAYY/Yj7fO7jGOwY/clip_image015_thumb%25255B1%25255D.png?imgmax=800" width="386" height="190"></a> <p>getRandom looks pretty much identical to the Haskell example: <p><a href="http://lh3.ggpht.com/-Y21mDMZ7NE8/T8-phIy3EWI/AAAAAAAAAYg/lXGP81UIHU8/s1600-h/clip_image016%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image016" border="0" alt="clip_image016" src="http://lh6.ggpht.com/--TqoTZZEn4I/T8-phpPQhvI/AAAAAAAAAYo/V-SLivnfi4Y/clip_image016_thumb%25255B1%25255D.png?imgmax=800" width="417" height="75"></a> <p>The composed version using the StateBuilder is shown above, but here’s the desugared version: <p><a href="http://lh5.ggpht.com/-A5Xq-qInCrc/T8-pkAKREGI/AAAAAAAAAYw/xettAFZm3NM/s1600-h/clip_image017%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image017" border="0" alt="clip_image017" src="http://lh4.ggpht.com/-kC6b0zkTQcw/T8-pu2lZIuI/AAAAAAAAAY4/Ju9beD-ivy4/clip_image017_thumb%25255B1%25255D.png?imgmax=800" width="362" height="180"></a> <p>The F# version is much clearer to read than the C#, in part due to the type inference meaning that the code has less pointless fluff. Having functions live in the module and not as part of a class aids that too (can simply call setState instead of State.Set in the C#) <p>Following through the simpler example really helped my understanding, and hopefully will see examples in the future where it will be sensible to use the state monad in my coding. Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com1tag:blogger.com,1999:blog-848045343259901678.post-18948598089077242282012-05-21T13:05:00.001-07:002012-06-06T12:22:25.929-07:00Monte Carlo C++ AMP<p>This blog is starting to look very inconsistent – my last blog post was talking about starting to write a game, and now I’ve gone onto a totally different topic. Due to time the game’s gone onto, if not the back burner, then a gentle simmer. I’ve got to balance doing <i>cool stuff</i> with keeping on top of various technologies that may be relevant as a contractor, and having a life! <p>This blog will describe using C++ AMP for calculating Pi; the calculation of Pi is a fairly good example of Monte Carlo calculations, as the algorithm’s simple, and we all know the result. <p>First off, this is what the algorithm looks like in C#: <p><a href="http://lh5.ggpht.com/-kQx-9Uu4cTg/T7qfxSkBSMI/AAAAAAAAASY/nJTF2NOAy7I/s1600-h/clip_image001%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image001" border="0" alt="clip_image001" src="http://lh4.ggpht.com/-Ig-ovcpbZv8/T7qfyd14tTI/AAAAAAAAASg/1EoyrpI7hJk/clip_image001_thumb%25255B1%25255D.png?imgmax=800" width="505" height="215"></a> <p>It simply finds the number of circles representing x-y coordinate pairs, whose vector magnitude falls within the unit circle. This ratio is Pi / 4. <p>The random class is the Mersenne Twister code from here: <a href="http://takel.jp/mt/MersenneTwister.cs">http://takel.jp/mt/MersenneTwister.cs</a>. The results are: <p><a href="http://lh3.ggpht.com/-SoQMLHpJhBU/T7qfzwwJeMI/AAAAAAAAASo/7EER67ufq-4/s1600-h/clip_image002%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image002" border="0" alt="clip_image002" src="http://lh6.ggpht.com/-LB1Tti57t2w/T7qf0z2EwvI/AAAAAAAAASw/XuM1MKEqUm8/clip_image002_thumb%25255B1%25255D.png?imgmax=800" width="317" height="257"></a> <p>Replacing the Mersenne Twister class with System.Random resulted in code which was approximately 30% slower. I’m running this on a dual-core laptop, but have not parallelised it, as I didn’t fancy porting over a Parallel Random Number Generator (PRNG) myself. <p>Tomas Petricek has an example of using the GPU to calculate Pi using F# <a href="http://tomasp.net/articles/accelerator-intro.aspx">here</a>, but his example generates the random numbers on the CPU and uploads them to the GPU to do the calculations and reduction. <h3>C++ AMP Version</h3> <p>Microsoft have just released a C++ AMP Random Number generator library to codeplex <a href="http://amprng.codeplex.com/">http://amprng.codeplex.com/</a>, but I’m using BharathM port of the parallel Mersenne Twister described here: <a href="http://blogs.msdn.com/b/nativeconcurrency/archive/2011/12/20/mersenne-twister-sample-using-c-amp.aspx">http://blogs.msdn.com/b/nativeconcurrency/archive/2011/12/20/mersenne-twister-sample-using-c-amp.aspx</a> <p>His example generates random numbers into a rank 2 array of size 4096 * 2, but I’ve modified g_n_per_RNG to be 256.</p> <p><a href="http://lh6.ggpht.com/-Ws31sNK1BIY/T7qf1w68LDI/AAAAAAAAAS4/wHFK92K4oqw/s1600-h/clip_image004%25255B4%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image004" border="0" alt="clip_image004" src="http://lh4.ggpht.com/-rhMcn6gvBm8/T7qf2u2P0DI/AAAAAAAAATA/k34Dafogf5Y/clip_image004_thumb%25255B1%25255D.jpg?imgmax=800" width="459" height="127"></a></p> <p>The first thing I do is to pair up those random numbers into x-y coordinates and to find the square of their magnitude: <p><a href="http://lh4.ggpht.com/-FmVf91kUpzQ/T7qf3TgEh7I/AAAAAAAAATE/I272yeEkLj8/s1600-h/clip_image006%25255B4%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image006" border="0" alt="clip_image006" src="http://lh4.ggpht.com/-J6MvPpRtscc/T7qf4ZB0VEI/AAAAAAAAATQ/zyo2U2fbfCs/clip_image006_thumb%25255B1%25255D.jpg?imgmax=800" width="465" height="121"></a> <p><a href="http://lh4.ggpht.com/-Da2AGDXj_FM/T7qf5IMd_eI/AAAAAAAAATY/6hQN1jTa5Tg/s1600-h/clip_image008%25255B4%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image008" border="0" alt="clip_image008" src="http://lh4.ggpht.com/-eak10-3pH6Y/T7qf6PuxYtI/AAAAAAAAATg/ElAHB23X2v0/clip_image008_thumb%25255B1%25255D.jpg?imgmax=800" width="466" height="170"></a> <p>Then I determine whether the magnitude of these coordinates fall outside of the unit circle: <p><a href="http://lh3.ggpht.com/-DJVHl5dJl38/T7qf6qeq1zI/AAAAAAAAATk/9WRap2koDUw/s1600-h/clip_image010%25255B4%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image010" border="0" alt="clip_image010" src="http://lh3.ggpht.com/-mtrahXt0ObQ/T7qf7VrDjBI/AAAAAAAAATs/DNCmPvpvYFM/clip_image010_thumb%25255B1%25255D.jpg?imgmax=800" width="467" height="77"></a> <p><a href="http://lh3.ggpht.com/-UG6rWtlQwmc/T7qf8esfPuI/AAAAAAAAAT0/imnNMQYkLaA/s1600-h/clip_image011%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image011" border="0" alt="clip_image011" src="http://lh3.ggpht.com/-fpcQOt1KmsQ/T7qf9Bkw-GI/AAAAAAAAAUA/swpaXfPQQng/clip_image011_thumb%25255B1%25255D.png?imgmax=800" width="466" height="191"></a> <p>The final step is to do a parallel reduce of the resulting array. This is using the just-released reduction method from the C++ AMP algorithms library: <a href="http://ampalgorithms.codeplex.com/">http://ampalgorithms.codeplex.com/</a> <p><a href="http://lh3.ggpht.com/-AZENML9vYsM/T7qf-MIU8jI/AAAAAAAAAUI/3BXXw5HTBpU/s1600-h/clip_image013%25255B4%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image013" border="0" alt="clip_image013" src="http://lh3.ggpht.com/-OiP749c7ats/T7qf_Yo_gvI/AAAAAAAAAUQ/ZQlSXir84gE/clip_image013_thumb%25255B1%25255D.jpg?imgmax=800" width="472" height="113"></a> <p>The reduce function behaves similar to STL’s std::accumulate in that you can pass in the binary operation to perform on each item. The function takes a rank 1 array, so I’m using view_as to change rank. <p>To perform the timings I run the algorithm twice, once to warm up (ensure all shaders etc are compiled – see <a href="http://social.msdn.microsoft.com/Forums/en-US/parallelcppnative/thread/1c6c9f04-1f9f-4f44-99f3-154d991ae5ba">http://social.msdn.microsoft.com/Forums/en-US/parallelcppnative/thread/1c6c9f04-1f9f-4f44-99f3-154d991ae5ba</a> ) <p>The results I obtained are: <p><a href="http://lh3.ggpht.com/-U4Jv3ibOzJI/T7qgAEyIlEI/AAAAAAAAAUY/DHFwhVkXV3U/s1600-h/clip_image015%25255B4%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image015" border="0" alt="clip_image015" src="http://lh6.ggpht.com/-XiTU0Nqi-6M/T7qgBAbji7I/AAAAAAAAAUg/0sCJ1wHMmNA/clip_image015_thumb%25255B1%25255D.jpg?imgmax=800" width="450" height="246"></a> <p>Calculating the result for half a million random numbers took approximately 16 milliseconds, whereas for the single threaded C# code a million iterations took 44 ms, so accounting for using both cores the performance seems pretty similar. This is quite disappointing, even though the GPU isn’t the most powerful (a mobile 5470), the CPU isn’t the most powerful either, so I was hoping for near an order of magnitude speed up. I wasn’t doing anything clever with tiling, and there may be other bits of the API I’m not using correctly. I’ll have to get hold of Kate Gregory’s <a href="http://shop.oreilly.com/product/0790145341907.do">book</a> and try with different examples.</p> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com3tag:blogger.com,1999:blog-848045343259901678.post-47210718476676817322012-04-14T13:36:00.001-07:002012-06-06T12:22:04.596-07:00Bullet Physics and DirectX 11 on Windows 8 Metro<p>I've got a game idea which I think may work on a tablet so I've created the following 'demo'. I've hooked together the Bullet Physics engine with some 3D graphics, and the falling wall of cubes could pretty much be the canonical "Hello, World" equivalent for a 3D game. I'm not sure how much time I'll have to pursue my game idea, but it should in the worst case be a route to learning some new technologies.</p> <p>The demo won't win any awards for its graphics, and the video is pretty poor quality (I recorded the screen with my camera instead of using any screen capturing software).</p> <p> <div style="padding-bottom: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; float: none; padding-top: 0px" id="scid:5737277B-5D6D-4f48-ABFC-DD9C333F4C5D:4954533f-17c2-4952-9b1b-f9999e54cbcc" class="wlWriterEditableSmartContent"><div id="70bb5816-76bf-4c9d-9ef4-4df7848b6a32" style="margin: 0px; padding: 0px; display: inline;"><div><a href="http://www.youtube.com/watch?v=zmD-7TfDVXk" target="_new"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheR2wQWLMOpuqsfCA5QBebopFZuztUqGoty6Lz9NMqtZ6-431187XfCxvTR0pI7b7LSPZ4Ymk_UGYjEZHp0XKw0GQZpKdt8h1IjphvvooWhz5YreffvqK8b0mp0oRiwwp0pwa58PLpSeA/?imgmax=800" style="border-style: none" galleryimg="no" onload="var downlevelDiv = document.getElementById('70bb5816-76bf-4c9d-9ef4-4df7848b6a32'); downlevelDiv.innerHTML = "<div><object width=\"480\" height=\"270\"><param name=\"movie\" value=\"http://www.youtube.com/v/zmD-7TfDVXk?hl=en&hd=1\"><\/param><embed src=\"http://www.youtube.com/v/zmD-7TfDVXk?hl=en&hd=1\" type=\"application/x-shockwave-flash\" width=\"480\" height=\"270\"><\/embed><\/object><\/div>";" alt=""></a></div></div><div style="width:480px;clear:both;font-size:.8em">Falling cubes</div></div></p> <p>Microsoft seem to be definitely pushing for the use of C++ for game creation with Windows 8, with no announcement of a new version of XNA, but that seems to make sense; C++ is closer to the metal than C# and will get more performance out of lower power ARM tablets. Also, there's quite a buzz with the C++ renaissance with C++11, and modern-style C++. The corresponding new technologies in VS 2010 (PPL) and VS 11 (auto-vectorization, C++ AMP) are especially cool.</p> <p>I'm also keen to get back into C++ - the past year or so I've been working purely in C#, but all my prior experience was using a mixture of C# and C++, and I do miss C++ (even though I am more productive in C#). Also, my previous roles were generally in a modern style (use of STL, smart pointers etc.) but my most recent experience was "C with Classes" style code, so I've been relearning modern style as well as picking up the new C++11 features.</p> <p>Building Bullet as a static lib was pretty easy, I just had to switch it over to use the Multi-threaded DLL c-runtime. Defining WINAPI_FAMILY=WINAPI_PARTITION_APP to detect any non-metro API usages revealed the usage of GetTickCount, which is unavailable on Metro apps, so I just turned off profiling by defining BT_NO_PROFILE 1 in btQuickprof.h</p> <p>The Windows 8 samples including a couple of games which are decent guides on using DirectX 11. These demos aren't supposed to be examples of a whole game engine, and don't necessarily follow best practices: Marble Maze has a class DirectXBase which encapsulates all rendering logic, which is great, but the game logic is in a class MarbleMaze, which derives from this class. It's nothing that would trip up an experienced developer, but beginners may follow the example. The Simple3DGame (which also demos DirectX and XAML interop) is better structured. The only strange thing in that example is that the render primitive classes are all C++/CX ref classes. This seems to contradict what I remember Herb Sutter saying in a Channel 9 video about sticking to ISO standard C++ where possible and only using C++/CX at the boundaries.</p> <p>I'm not sure yet whether I'll try to continue to create a simple demo of my game idea purely in C++, or create a C++/CX interop layer to facilitate writing the game logic in C# - I'm tempted to do this as I'm more productive in C#, and it'll probably lead to a deeper understanding of WinRT which will be useful even when creating C# XAML applications.</p> <p>I could have alternatively stayed purely in C# and used SharpDX, or a game framework such as ANX or Ogre (both of which say will soon be supporting XAML), but I was keen on having control of the whole game, C++ upwards - if after profiling any of the C# code any bottlenecks are found, it'll be easier to re-implement those parts of the code in C++.</p> <p>As to my experiences of developing the demo: The new version of C++ definitely feels more productive, and feels closer to C#. I love auto (var), lambdas (pretty much the same), for_each (foreach) and range for, and initializer lists. Also, async programming using PPL tasks feels very similar to .NET's TPL. I've had to change mental gears a bit in thinking again about object ownership instead of relying on the garbage collector, and it took way longer than it should for me to figure out how to add a std::unique_ptr into a vector.</p> <p>I could share the demo code if anyone would find it useful (I need to tidy it up a little bit, and can't be bothered otherwise) - just post in the comments!</p> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com6tag:blogger.com,1999:blog-848045343259901678.post-60487178826688451762012-03-12T14:38:00.001-07:002012-03-24T03:23:27.497-07:00Messing with Direct2D<p>As I blogged about <a href="http://taumuon-jabuka.blogspot.com/2012/01/charting-performance.html">previously</a>, the main bottleneck in the oscilloscope code was in the drawing of the charts, in my previous post I modified DynamicDataDisplay to optimise this, but I wanted to make a Metro version of my app, and didn’t want to convert that library over, so I thought I’d look at doing the drawing myself.</p> <p>First off, on Windows 7/.NET4 I used the <a title="http://windowsclient.net/blogs/rob_relyea/archive/2010/04/30/gizmodo-posts-wpf-direct2d-sample-wow.aspx" href="http://windowsclient.net/blogs/rob_relyea/archive/2010/04/30/gizmodo-posts-wpf-direct2d-sample-wow.aspx">SurfaceInteropHelper</a> which allows Direct2D to be hosted on a WPF control, to do the drawing, and found that the CPU usage dropped from 70% to just under 40% (the code’s not worth showing here, and my use of Direct2D was pretty simplistic).</p> <p>For the Windows 8 consumer preview, I created an app that used the <a title="http://msdn.microsoft.com/library/windows/apps/Hh702626" href="http://msdn.microsoft.com/library/windows/apps/Hh702626">SwapChainBackgroundPanel</a> . It wasn’t obvious how to do this in an app created from scratch, so I butchered the SDK sample <a title="http://code.msdn.microsoft.com/windowsapps/Metro-style-DirectX-18f98448" href="http://code.msdn.microsoft.com/windowsapps/Metro-style-DirectX-18f98448">Metro style DirectX shooting game sample (XAML)</a></p> <p>This control allows Direct2D and XAML to be mixed in a metro app, but the control needs to be at the root element. I can see how this would be OK for an app such as a game where it could make use of overlayed controls, but for an app such as mine where I’d like to have multiple charts drawn using Direct2D it’s not ideal. Also, it seems to force the skeleton of the app to be written in C++ (where it would be nice to have a C# app host the controls, with the C++ –hidden- ).</p> <p>This was awkward, but not a blocker to implementing the app on Metro… the blocker was to do with accessing the microphone. There are new Media Foundation libraries to do that, which do interoperate with the new C# async features than DirectShow (expose awaitable types):</p> <p><a href="http://lh6.ggpht.com/-bUfSFewDt6A/T15suU5-6wI/AAAAAAAAARQ/Mj4NiJ7ZzRM/s1600-h/image%25255B3%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-ri-ZLeYcidc/T15svEJVhrI/AAAAAAAAARY/Ai6cPUxNjHg/image_thumb%25255B1%25255D.png?imgmax=800" width="479" height="164"></a></p> <p>The new version of RX (2.0 beta) has a new CreateAsync method instead of having an overload of Create that takes the async lambda).</p> <p>The code above is untested, as the blocker is in the new MediaCapture library:</p> <p><a href="http://lh6.ggpht.com/-nH7HmeY9MA0/T15sweKGT_I/AAAAAAAAARg/G5op3aXPBwQ/s1600-h/image%25255B7%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-XSHYLWhL51Y/T15sxHI73GI/AAAAAAAAARk/t9V8Hu1kjiU/image_thumb%25255B3%25255D.png?imgmax=800" width="480" height="171"></a></p> <p>There’s no way to create a lossless MediaEncodingProfile from which to extract the raw waveform.</p> <p>I think I’ll leave this app on the backburner until at least a later beta.</p> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com1tag:blogger.com,1999:blog-848045343259901678.post-28404024067505470452012-02-15T13:10:00.001-08:002012-03-24T03:23:05.540-07:00TPL Dataflow<p>This blog post is coming a couple of weeks late due to my copy of Windows becoming corrupted – and strangely the only things I didn’t have backed up were this Oscilloscope app! <p>Anyway, last time I took a look at <a href="http://taumuon-jabuka.blogspot.com/2012/01/charting-performance.html">charting performance</a>, but this post will investigate <a href="http://channel9.msdn.com/Shows/Going+Deep/Stephen-Toub-Inside-TPL-Dataflow">TPL Dataflow</a>. <p>In my original <a href="http://taumuon-jabuka.blogspot.com/2012/01/visualising-sound.html">F# post</a>, I discussed that I felt that the code would have been better implemented in RX, which I then went and did in <a href="http://taumuon-jabuka.blogspot.com/2012/01/oscilloscope-using-rx-and-c-async-ctp.html">this post</a>. However, RX isn’t the only recent new technology coming out of Microsoft that deals with streams of data. TPL Dataflow has a confusingly similar overlap with the usages of RX, and this is what its whitepaper has to say: <p>“Astute readers may notice some similarities between TPL Dataflow and Reactive Extensions (Rx), currently available as a download from the MSDN Data developer center. Rx is predominantly focused on coordination and composition of event streams with a LINQ-based API, providing a rich set of combinators for manipulating IObservable<T>s of data. In contrast, TPL Dataflow is focused on providing building blocks for message passing and parallelizing CPU- and I/O-intensive applications with high-throughput and low-latency, while also providing developers explicit control over how data is buffered and moves about the system. As such, Rx and TPL Dataflow, while potentially viewed as similar at a 30,000 foot level, address distinct needs. Even so, TPL Dataflow and Rx provide a better together story.” <p>That does sound very interesting – who doesn’t want better performance! <h3>Implementation</h3> <p>I’ll dive in straight away and look at some code. The code is structured similarly to the RX code in my previous post. <p><a href="http://lh6.ggpht.com/-zjXUXt7LX48/Tzwe3Fj5mtI/AAAAAAAAAN8/WeSa5BGixIw/s1600-h/clip_image001%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image001" border="0" alt="clip_image001" src="http://lh4.ggpht.com/-7gtLkWIzJRE/Tzwe37Ks0AI/AAAAAAAAAOE/iWnFvovnVbY/clip_image001_thumb%25255B1%25255D.png?imgmax=800" width="477" height="177"></a> <p>The microphone still provides an IObservable<float[]> via the GetStream() method. The TaskSchedulers are specified explicitly throughout, so that the code can be unit tested (I’m using a <a href="http://julmar.com/blog/mark/?p=9">MockSynchronizationContext</a>). <p>The transformManyBlock behaves in the same way to the SelectMany operator, it separates the float array into its constituent parts. <p>The broadcast block is analogous to a connectable observable – many blocks can be linked to it to receive the same messages. <p>The transform many block is then explicitly linked to the broadcast block, meaning that its outputs make it into the broadcast block’s input. <h4>Oscilloscope/Buffered View</h4> <p>The code to deal with the buffered display of data is: <p><a href="http://lh4.ggpht.com/-ghGaHihGWHA/Tzwe4gDVFlI/AAAAAAAAAOM/rKqnYvpIXRk/s1600-h/clip_image002%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image002" border="0" alt="clip_image002" src="http://lh4.ggpht.com/-o3K_WO0LCUE/Tzwe5Ry6OUI/AAAAAAAAAOU/4fZwbA7swUs/clip_image002_thumb%25255B1%25255D.png?imgmax=800" width="498" height="189"></a> <p>The batch block behaves the same as LINQ’s Buffer operator (a bit confusingly when TPL Dataflow’s Buffer means something different). This simply takes the incoming float values and packages them into an array. <p>The ActionBlock is the final point on the chain – it takes the values it receives and outputs them to the screen (via oscilloscopeOutput) – this is the equivalent code to the code in the Observable Subscription in the RX version of my code. <h4>Sliding Window View</h4> <p>Again, the sliding window view is structured similarly to the previous RX code: <p><a href="http://lh5.ggpht.com/-GwoyBbzHhqc/Tzwe6Kfwm4I/AAAAAAAAAOc/D7u1dzEI6f8/s1600-h/clip_image003%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image003" border="0" alt="clip_image003" src="http://lh5.ggpht.com/-CHyRpvQqIXI/Tzwe7OykY2I/AAAAAAAAAOk/FbcXOk0I4aI/clip_image003_thumb%25255B1%25255D.png?imgmax=800" width="462" height="248"></a> <p>A sample block is created so that every <i>n</i> samples are fed into the SlidingWindowBlock. This returns a new float[] array representing the new window of data on every input. <p>The second sample block ensures that not every window of data is drawn (so that the chart is refreshed at 40fps rather than 400fps!) <p>Finally, the observable is subscribed to, and the messages are posted into the transformManyBlock: <p><a href="http://lh6.ggpht.com/-yxcQf4GRDQQ/Tzwe9fXcuSI/AAAAAAAAAOs/TncQ2YwqaFc/s1600-h/clip_image004%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image004" border="0" alt="clip_image004" src="http://lh4.ggpht.com/-M6QYj1stw0Q/Tzwe-2xkK-I/AAAAAAAAAO0/LTjB-OBmP0w/clip_image004_thumb%25255B1%25255D.png?imgmax=800" width="435" height="63"></a> <p>Whereas the RX guidelines recommend creating new operators based on existing operators, TPL Dataflow encourages building up custom blocks (though they can be built out of combining existing blocks). <p>The SlidingWindowBlock is a slightly modified version of the one provided in the white paper, and here (<a href="http://msdn.microsoft.com/en-us/library/hh228606(v=vs.110).aspx)">http://msdn.microsoft.com/en-us/library/hh228606(v=vs.110).aspx)</a> The one in the example only posts a window once it has received a full window worth of data, whereas I want to start drawing the windows as soon as data arrives. <p><a href="http://lh6.ggpht.com/-Pt5828m3ARY/TzwfBFa1byI/AAAAAAAAAO8/HeKB9XPn4rA/s1600-h/clip_image005%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image005" border="0" alt="clip_image005" src="http://lh4.ggpht.com/-3YRIVrgm8u8/TzwfB-hQEWI/AAAAAAAAAPA/Zg_zqBmjmCM/clip_image005_thumb%25255B1%25255D.png?imgmax=800" width="459" height="264"></a></p> <p>The SampleBlock is simple, it yields a value once it has received enough inputs, discarding the intermediate ones. This is structured more efficiently than my RX implementation – that one buffers the values, and then only take the last value out of the buffer.</p> <h4><a href="http://lh3.ggpht.com/-rjQXmXrYT8c/TzwfCvWQo8I/AAAAAAAAAPM/Aqdcfc4cb1I/s1600-h/clip_image006%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image006" border="0" alt="clip_image006" src="http://lh4.ggpht.com/-ynmuqoAvuUg/TzwfE3x8FsI/AAAAAAAAAPU/KetUltmgkOg/clip_image006_thumb%25255B1%25255D.png?imgmax=800" width="436" height="234"></a></h4> <h4>Performance</h4> <p>I removed the updating of the chart, to look purely at what the performance of the app is in taking the input from the microphone, and shaping it to be in the correct format for output. <p>RX Performance: <p><a href="http://lh6.ggpht.com/-qfafqdhNDB4/TzwfFhsDIUI/AAAAAAAAAPc/LTaRDCCB8D8/s1600-h/clip_image007%25255B5%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image007" border="0" alt="clip_image007" src="http://lh4.ggpht.com/-YGUTP4SpHY8/TzwfHs4xXXI/AAAAAAAAAPk/hTqPKrO7dkc/clip_image007_thumb%25255B2%25255D.png?imgmax=800" width="306" height="522"></a> <p>TPL Dataflow: <p><a href="http://lh6.ggpht.com/-vI-VmLDJcN0/TzwfI1E-zOI/AAAAAAAAAPs/8V_IARdlFQU/s1600-h/clip_image008%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image008" border="0" alt="clip_image008" src="http://lh5.ggpht.com/-4iqHcQHUWgc/TzwfLEtY4yI/AAAAAAAAAP0/mLH5MvI_658/clip_image008_thumb%25255B1%25255D.png?imgmax=800" width="308" height="510"></a> <p>The processor usage between the apps is in the same ballpark, but the TPL Dataflow app has many more garbage collections, and # bytes in all heaps also is running slightly higher. I had a quick look in the profiler, and it seems that there are many Task allocations from the Dataflow.Post() operation from within the SlidingWindow and Sample implementations. <p>The extra generation 1 collections don’t really matter for this application with the functionality that it currently has – the % time in GC barely registers, and there are no Gen 2 garbage collections over the minute or so that the app was running. <p>Once the app is doing more calculations and drawing more complex charts it would be interesting to see whether any latency spikes due gen 2 GCs cause similar slowdowns to the one discussed at the start of my previous post. <p>It would be an interesting exercise to limit the amount of GCs throughout the app, for instance there’s no need for the microphone access observable to return an IObservable<float[]> instead of IObservable<float>; currently every read from the microphone allocates a new float[]. Similarly, new List<Point> are created to more easily interface with DynamicDataDisplay – it would be better to change the types of data that D3 takes to be more observable-friendly, and to save having so many allocations. Again, there’s not much point doing this, other than an interesting theoretical exercise, until the garbage collection overhead proves to be a performance issue. <h5>Conclusion</h5> <p>For an application as simple as this, there isn’t any benefit to using TPL Dataflow – it is a powerful library, with functionality such as blocks being able to decline offered blocks, and request them later, which would be difficult to implement in RX. As my app doesn’t (currently) need that level of functionality, there’s no benefit to using the library. <p>I may revisit this in the future – if I had some computationally expensive operation (FFT for instance) where I’d want greater control over the flow of data through the system.</p> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-51427003865253777262012-01-25T13:12:00.001-08:002012-02-15T13:14:11.589-08:00Charting Performance<p>I wasn’t going to blog about performance until a later post, but there was a performance issue I wanted to tackle before discussing some <i>other things.</i></p> <p>The frame rate sometimes seemed to be as expected, but sometimes slow, and the application was quite unresponsive when handling the Stop button command.</p> <p>To help nail this down I created a Buffer Frame Rate performance counter, so that I monitor it along with another couple of metrics (I chose to see the count of gen1 and gen2 garbage collections, along with the % time in GC, and % processor time).</p> <p><a href="http://lh5.ggpht.com/-prMYYji_cg8/TyBwAnV-meI/AAAAAAAAAL8/AiHmdGyjPMc/s1600-h/image%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-lHoHjOoqrkY/TyBwBSLUehI/AAAAAAAAAME/yeFgP4gyQic/image_thumb%25255B2%25255D.png?imgmax=800" width="371" height="473" /></a></p> <p>The framerate average isn’t bad, but the frame rate is not consistent; it regularly drops below 10 frames per second. This post will discuss optimising the performance of the application to keep the frame rate steady and ensure that the app is responsive.</p> <p>Performance increases should generally focus on reducing unnecessary work in the whole application, and not micro-optimisations. In this case, it’s realising that the top chart is drawing at 40 frames per second (a buffer of 1000 data points into a 40KHz source), whereas the bottom chart is updating at 400 frames per second (sampling the data to 100 points, and updating the chart on each of those sample points).</p> <p>However, as there are other types of charts I want to produce it’s worth investigating where the time is spent.</p> <p>First off, I tried turning off anti-aliasing, and moving over to the d3future project (<a href="http://d3future.codeplex.com">http://d3future.codeplex.com</a> ), to see if that made any performance difference, but the performance was pretty similar, with the CPU pegged at 100%. A further run generated this interesting trace:</p> <p><a href="http://lh4.ggpht.com/-0xat8MNOeGE/TyBwCE8ruQI/AAAAAAAAAMM/GxIi_m8LlVo/s1600-h/image%25255B8%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/--zysqrqhf90/TyBwC4NoNqI/AAAAAAAAAMU/G1TT8ML99eU/image_thumb%25255B4%25255D.png?imgmax=800" width="434" height="489" /></a></p> <p>The frame count starts initially high, but after a while performance drops off dramatically, which is followed by some long spikes in the % Time in GC. The drop-off maybe seems to correspond with one of the gen 2 collections. This situation wasn’t so reproducible that I could catch it under a profiler.</p> <p>My next step was to profile to see where the application was spending its time, just using the Visual Studio profiler. The application’s hot path was in the D3 LineChart’s UpdateCore method, where it was updating the chart’s Bounds using data from the EnumerableDataSource –it was iterating over the whole collection to find the x and y min and max values (to fit the chart to the data area).</p> <p>It seems unnecessary to iterate over the whole data source to get the minimum and maximum x values – for a linear chart (i.e. not a scatter chart) it would be expected that the first point is at the minimum x value and the last point the maximum.</p> <p>I created a chart derived from LineChart where I could instead pass in a List of points for the datasource – this means that I could get the last point without enumerating the whole collection, allowing the min and max x values to be quickly found. For the y values, I happen to want the chart to be a fixed axis and not scale to fit the data (instead scaled by a user-configurable gain factor), so there was no need at all to iterate over the collection.</p> <p>The chart looked like this after those changes:</p> <p><a href="http://lh3.ggpht.com/-GoQ_3w--bhU/TyBwDXsN4MI/AAAAAAAAAMY/uthIfjDXG1M/s1600-h/image%25255B12%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-cXH0dq80bhc/TyBwD2rgxAI/AAAAAAAAAMg/joBeEkG8Fy0/image_thumb%25255B6%25255D.png?imgmax=800" width="318" height="484" /></a></p> <p>The frame rate is pretty consistent now. The gen 1 allocations are still occurring at a rate of 200 per minute, but the application is now responsive to the stop click, stopping immediately.</p> <p>Profiling with memory allocations turned on showed that the types with most instances was System.Byte[] with 32% of the instances allocated in the profiling period.</p> <p>ConvertByteArrayToFloatArray() previously looked like this:</p> <p><a href="http://lh4.ggpht.com/-lHajzBLahWk/TyBwEXnKBdI/AAAAAAAAAMo/17EgLOzl3PM/s1600-h/image%25255B17%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-c_bbN-uFmB8/TyBwFIS7TeI/AAAAAAAAAMw/8pJGN-dshZg/image_thumb%25255B9%25255D.png?imgmax=800" width="522" height="138" /></a></p> <p>And after changing this to not use LINQ:</p> <p><a href="http://lh3.ggpht.com/-ldV6xDbM39Y/TyBwFU1NwYI/AAAAAAAAAM0/jItPq1L31fg/s1600-h/image%25255B22%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-kQVKD5Ie5Ys/TyBwGLS9v8I/AAAAAAAAANA/M-oZoJb29F4/image_thumb%25255B12%25255D.png?imgmax=800" width="562" height="162" /></a></p> <p>Gives the following:</p> <p><a href="http://lh4.ggpht.com/-Uvv-bTInSuI/TyBwGpgdPQI/AAAAAAAAANM/J1jQzWT1uRc/s1600-h/image%25255B27%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-HWs9lbY3M4Y/TyBwHlhJtOI/AAAAAAAAANU/wmdfgpmThco/image_thumb%25255B15%25255D.png?imgmax=800" width="406" height="574" /></a></p> <p>Performance is pretty similar to previous, but the gen 1 collections are occurring at a slightly lower rate. Now, profiling tells me that the types with most instances allocated is pretty much split between System.Action, System.Object, and System.Reactive.Disposables.SingleAssignmentDisposable.</p> <p>Profiling at this point identifies the majority of the work now being in LineGraph.UpdateCore() where its transforming all points from their data coordinates into screen coordinates.</p> <p>The transformation of coordinates for many points is embarrassingly parallel (think GPGPU). Potentially, the GPU could be put to use by simply drawing the points untransformed, adding a transformation onto the drawing context.</p> <p>Further performance increases could be done for the sliding window chart: A sliding window chart redisplays the same points again and again, as they move left along the x axis. It essentially transforms the same point many times into the new viewport. Doing a transform to move an existing point to the left could be more efficient than transforming again from data to screen coordinates.</p> <p>This blog so far has looked at some micro-optimisations focussing on identifying areas where work can be reduced. As I said in the beginning, if I wasn’t so interested in optimising for future requirements, I’d simply ensure that I was doing less drawing straight away.</p> <p>Anyway, the sliding observable can be change to only take every 10th windowed set of values, by introducing another Sample() on the result of the WindowWithCount():</p> <p><a href="http://lh3.ggpht.com/-ZuAamKDINmE/TyBwIGXy2wI/AAAAAAAAANY/fJh9NQNpK5M/s1600-h/image%25255B31%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-oe4Um_VsBq0/TyBwIwgz2bI/AAAAAAAAANk/G-PFeRlLVmM/image_thumb%25255B17%25255D.png?imgmax=800" width="591" height="310" /></a></p> <p>There isn’t actually that much difference in real-world performance.</p> <p><a href="http://lh4.ggpht.com/-FzbXeOPSMBM/TyBwJqGVQKI/AAAAAAAAANs/1EgSTWVXEUA/s1600-h/image%25255B36%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-XZbIWTyDW9Q/TyBwKZ6St2I/AAAAAAAAAN0/_1PWYujkYiU/image_thumb%25255B20%25255D.png?imgmax=800" width="415" height="640" /></a></p> <p>The rate of gen 1 collections is lower still than previous, but the profiler shows that app is still spending the majority of its time in drawing the charts. Surprisingly the CPU usage doesn’t seem to have dropped much, although in the profiler the time can now be seen to be split between the drawing of the two charts, instead of dominated by the drawing of the sliding window chart.</p> <p>The micro-optimisations were still worth investigating, as I have charts in mind that will draw multiple traces simultaneously.</p> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-60385307478875685722012-01-18T12:10:00.001-08:002012-03-24T03:23:05.541-07:00Oscilloscope using RX and C# Async CTP<p>In my last blog post I described the implementation of a simple ‘oscilloscope app’ in F#, as I wanted to see how the code would be structured using only F# idioms (<a href="http://taumuon-jabuka.blogspot.com/2012/01/visualising-sound.html">http://taumuon-jabuka.blogspot.com/2012/01/visualising-sound.html</a> )</p> <p>My natural instinct would have been to implement it using the Reactive Extensions for .NET (RX Framework), but I first wanted to investigate a pure F# solution. This post describes an alternate RX implementation.</p> <p>Similar to my last post, I’ll describe the code inside-out.</p> <h3>Creating the Observable</h3> <p><a href="http://lh5.ggpht.com/-eAycP7Dcv6A/Txcm6ixX81I/AAAAAAAAAK8/5V9V_w5mY7A/s1600-h/clip_image002%25255B7%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image002" border="0" alt="clip_image002" src="http://lh5.ggpht.com/-S5XNUHZL7XA/Txcm7pK32wI/AAAAAAAAALA/xSNg3KCK0Nc/clip_image002_thumb%25255B4%25255D.jpg?imgmax=800" width="599" height="471" /></a></p> <p>This code returns an IObservable with a float array payload, representing each read from the CaptureBuffer. The implementation could have internally started a dedicated Thread from which to push out values, but instead I’m using the new C# 5 async functionality (using the Async CTP Update 3), so that my code looks pretty similar to the previous F# example.</p> <p>The observable takes a CancellationToken, whose IsCancellationRequested is set to true once all subscriptions to the observable have been disposed. The CompositeDisposable returned out from the lambda is also disposed of at that point.</p> <p>The code loops while its CancellationToken has not been cancelled, asynchronously awaiting for one of the WaitHandles to be set, and then it reads from the buffer. The value is pushed out to subscribers in the OnNext.</p> <p>The ConvertByteArrayToFloatArray() is trivial:</p> <p><a href="http://lh5.ggpht.com/-6t3Ii3C8lLA/Txcm8O-tzvI/AAAAAAAAALE/zu3VDmMOao0/s1600-h/clip_image003%25255B5%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image003" border="0" alt="clip_image003" src="http://lh3.ggpht.com/-V_Q4vqmPU8M/Txcm87HsbbI/AAAAAAAAALI/SFXteKVI_KU/clip_image003_thumb%25255B2%25255D.png?imgmax=800" width="493" height="130" /></a></p> <h3>Subscriptions to the observable</h3> <p><a href="http://lh5.ggpht.com/-EB6Xl75B-lw/Txcm9Ri8ZfI/AAAAAAAAALM/a_5WOAby514/s1600-h/clip_image004%25255B6%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image004" border="0" alt="clip_image004" src="http://lh6.ggpht.com/-SE34OuUdWF8/Txcm-O0eiPI/AAAAAAAAALQ/EDaVXlSSUUA/clip_image004_thumb%25255B3%25255D.png?imgmax=800" width="510" height="101" /></a></p> <p>First off, the float array returned from the observable is decomposed into the individual values, using a <i>SelectMany</i>, as sampling, buffering and windowing operations all operate on a stream of floats. Then, the observable is published to an IConnectableObservable. The microphone access returns a Cold observable, meaning that each subscriber to it would end up creating their own microphone access. This would work (the CaptureBuffer doesn’t prevent this), but the connectable observable means that instead all clients share the same observable, and ensures that they all see the same values (so that the traces on the two charts are in sync).</p> <p>The RefCount() means that when all subscriptions to the <i>observable</i> IConnectableObservable variable have been disposed of then the subscription to the underlying observable will also be disposed.</p> <p><a href="http://lh6.ggpht.com/-jCL2vDpkNXo/Txcm-0RL2mI/AAAAAAAAALU/BvFq2YyuFwU/s1600-h/clip_image005%25255B5%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image005" border="0" alt="clip_image005" src="http://lh6.ggpht.com/-AZOf5bPqN9s/Txcm_nn49gI/AAAAAAAAALY/CaJ1_-1PNNE/clip_image005_thumb%25255B2%25255D.png?imgmax=800" width="541" height="205" /></a></p> <p>The top ‘oscilloscope’ trace is a simple Observable.Buffer() over the data stream. There is no need to <i>ObserveOn</i> the dispatcher thread as the subscription occurs on the UI thread. Using RX, it would be easy to schedule various parts of the work onto different threads, but I’ll discuss this in a later article (I want to keep everything on the UI thread for now to compare performance with the single threaded F# implementation).</p> <p>All subscriptions are added to a CompositeDisposable member in the ViewModel – the Stop button’s command implementation disposes of this, which causes all subscriptions to be disposed of, and so the microphone access loop to be terminated via its CancellationToken.</p> <p><a href="http://lh4.ggpht.com/-bZhPhDgghn0/TxcnApTW_NI/AAAAAAAAALc/cCsFEngfE5Y/s1600-h/clip_image006%25255B5%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image006" border="0" alt="clip_image006" src="http://lh6.ggpht.com/-udt4kq7bENw/TxcnCEYSFuI/AAAAAAAAALg/krsL9g6jGzk/clip_image006_thumb%25255B2%25255D.png?imgmax=800" width="567" height="250" /></a></p> <p>The windowing operation simply samples every 100 data points, and from those sampled data points takes a sliding window of 1000 values to display. The windowCount variable is closed over to allow the y axis to be continually updated.</p> <p><a href="http://lh3.ggpht.com/-CkBBC9GQSoQ/TxcnC7AAbxI/AAAAAAAAALk/zw-k8yB7LJY/s1600-h/clip_image007%25255B6%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image007" border="0" alt="clip_image007" src="http://lh3.ggpht.com/-cd5RhsRXya4/TxcnD53Mc9I/AAAAAAAAALo/2lbvYr78kZQ/clip_image007_thumb%25255B3%25255D.png?imgmax=800" width="471" height="164" /></a></p> <p>The Sample operator is simple, but not particularly efficient – it takes a buffer (i.e a non-overlapping window) of <i>count </i>values, and then takes the last value in the buffer.</p> <p>The WindowWithCount operator is the same one I discussed at <a href="http://taumuon-jabuka.blogspot.com/2011/07/rx-framework-performance-awareness.html">http://taumuon-jabuka.blogspot.com/2011/07/rx-framework-performance-awareness.html</a> (with implementation grabbed from <a href="http://social.msdn.microsoft.com/Forums/en-US/rx/thread/37428f58-f241-45b3-a878-c1627deb9ac4#bcdc7b79-bbde-4145-88e4-583685285682">http://social.msdn.microsoft.com/Forums/en-US/rx/thread/37428f58-f241-45b3-a878-c1627deb9ac4#bcdc7b79-bbde-4145-88e4-583685285682</a> )</p> <p><a href="http://lh4.ggpht.com/-4uheJ0pwzP4/TxcnEdmDvfI/AAAAAAAAALs/TA24oDx6St8/s1600-h/clip_image008%25255B5%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image008" border="0" alt="clip_image008" src="http://lh3.ggpht.com/-zyoDyBo4ln0/TxcnFp4F20I/AAAAAAAAALw/NDbhvztr-lM/clip_image008_thumb%25255B2%25255D.png?imgmax=800" width="577" height="243" /></a></p> <p>And as I also talked about in that post, the RX guidelines recommend implementing an operator in terms of existing operator. There’s only one problem in this case, it’s quite slow, (I’ll get quantitative figures on this in a future blog post discussing performance of all approaches).</p> <p>The following is faster (again, I know more specific figures are needed):</p> <p><a href="http://lh4.ggpht.com/-rwo_mA39Gfo/TxcnHCYs87I/AAAAAAAAAL0/4FZvEso1Has/s1600-h/clip_image009%25255B6%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image009" border="0" alt="clip_image009" src="http://lh3.ggpht.com/-JKKLsu0tqy8/TxcnIStGmCI/AAAAAAAAAL4/ovg1lfMqM2Y/clip_image009_thumb%25255B3%25255D.png?imgmax=800" width="514" height="340" /></a></p> <h3>Comparing implementations</h3> <p>For me, the RX solution is cleaner than the F# solution, and easier to follow. I did implement my F# solution in quite an imperative way though, and should have perhaps used AsyncSeq or mailbox processors, but as reading from the microphone is a push-based activity, none of those solutions would be as clean as RX (of course I haven’t covered using RX in F#). The F# version is much faster, and I’ll take a big more of a dig into performance in an upcoming blog post.</p> Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com2tag:blogger.com,1999:blog-848045343259901678.post-62128038825097031802012-01-10T14:38:00.001-08:002012-02-15T13:13:09.092-08:00Visualising Sound<p>This is the first in a series of blog posts about visualising sound, in a similar way to an oscilloscope. The geek in me thought it’d be a fun thing to do, as well investigate different technological approaches to implementing the app (interesting as it’s a real-time performant app). </p> <p><a href="http://lh3.ggpht.com/-XAYRLeQsY3o/Twy94FZe95I/AAAAAAAAAHo/9zfAHCbWY9A/s1600-h/clip_image001%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image001" alt="clip_image001" src="http://lh6.ggpht.com/-qX4HS0ris9s/Twy94lcjPCI/AAAAAAAAAHw/cXFKtMeV1TI/clip_image001_thumb%25255B1%25255D.png?imgmax=800" border="0" height="345" width="515" /></a></p> <p>An oscilloscope has a single screen, which refreshes on a given time period, displaying a number of traces. The user can control that time period, as well as the gain (the y axis scale).</p> <p>The top graph is essentially the same view as an oscilloscope containing a single trace, and a fixed time period (further blog posts may investigate varying time periods).</p> <p>The bottom graph is a sliding window with a longer time period – this is the advantage of implementing an oscilloscope in code, we can create charts that aren’t really feasible in a classic CRT oscilloscope.</p> <p>This series of blog posts will investigate implementing this screen using F# idioms, as well as using the Reactive Extension for .NET (RX Framework), and TPL Dataflow.</p> <p>There are further things that could be implemented in future blog posts which may be interesting to see how the varying approaches look like:</p> <p>· Trigger, or Trigger and hold: Only refresh the oscilloscope view once a certain trigger level has been reached. This is interesting as may want to include a certain number of immediately prior to the point where the trigger was set.</p> <p>· Log many traces.</p> <p>· Spectrum analyser/FFT.</p> <p>· Digital filtering.</p> <p>· Comparing traces, or more complicated math channels.</p> <p>· Heatmap.</p> <p>· Adjustable time offset (delay) – useful on the oscilloscope view to centre a waveform on the screen, or for when comparing two or more channels output.</p> <h3>F# Implementation</h3> <p>This blog post covers an F# implementation of the microphone, using asynchronous workflows. The graphing is being done by a nightly build download of <a href="http://dynamicdatadisplay.codeplex.com/">Dynamic Data Display (D3)</a>. I’m really impressed with the performance, but it is quite difficult to customise.</p> <p>The low-latency microphone code is from this article on gamedev (<a title="http://www.gamedev.net/topic/586706-slimdx-directsound-capture-microphone-stream-with-low-latency-example/" href="http://www.gamedev.net/topic/586706-slimdx-directsound-capture-microphone-stream-with-low-latency-example/">http://www.gamedev.net/topic/586706-slimdx-directsound-capture-microphone-stream-with-low-latency-example/</a>).</p> <p>The implementation of this is pretty simple; I’ll start from the detail out.</p> <p>The inner loop of the program is an asynchronous workflow that reads from the buffer and returns a sequence:</p> <p><a href="http://lh3.ggpht.com/--2pBXN42PBA/Twy95opEFRI/AAAAAAAAAH0/6Beo_KDDlD0/s1600-h/clip_image003%25255B5%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image003" alt="clip_image003" src="http://lh6.ggpht.com/-XfomP2_0mM4/Twy96Fq1dFI/AAAAAAAAAIA/KGyj9rBsCKo/clip_image003_thumb%25255B2%25255D.jpg?imgmax=800" border="0" height="346" width="607" /></a></p> <p>Note the slightly-strange looking code:</p> <p>let task = System.Threading.Tasks.Task<int>.Factory.StartNew(fun () -> WaitHandle.WaitAny(waitHandles))</p> <p>F# has an Async.AwaitWaitHandle() method, which unfortunately only waits on a single handle. We want to wait on both handles so that we get notified when the buffer is full every 4096 instead of every 8192 bytes. With 4 bytes per sample, and a sample rate off 44KHz, this is equivalent to getting notified at an approximate rate of 40 times per second instead of 20 times per second.</p> <p>I could have implemented Async.AwaitAnyWaitHandle() taking an array of WaitHandles, but looking at the code in the F# PowerPack, the code was quite complex. So, the code instead creates a new future to do the waiting and let us know which WaitHandle was set (this does mean that we’ve got the minor overhead of scheduling a new task to run on the task pool).</p> <p>The Async.StartImmediate method ensures that the ProcessStream call is marshalled back onto the UI thread. It may be worth in the future looking at doing more of the data processing on a dedicated thread, leaving the UI thread free for drawing and user input.</p> <p>The convertByteArrayToSequence is simple, it just iterates over the buffer in 4 byte chunks, and converts the values to floats, which it yields in the sequence:</p> <p><a href="http://lh5.ggpht.com/-fBC8G3ZE-Hg/Twy964_5WbI/AAAAAAAAAII/codL6Szmdpo/s1600-h/clip_image004%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image004" alt="clip_image004" src="http://lh4.ggpht.com/-6wHf1z92QH4/Twy97rnlaDI/AAAAAAAAAIQ/WwIdIBKo_mQ/clip_image004_thumb%25255B1%25255D.png?imgmax=800" border="0" height="62" width="524" /></a></p> <p>The ProcessStream method looks like this:</p> <p><a href="http://lh5.ggpht.com/--_3QdWf1-40/Twy98QMLbCI/AAAAAAAAAIU/1kvlxGVTxtk/s1600-h/clip_image006%25255B4%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image006" alt="clip_image006" src="http://lh6.ggpht.com/-rcm-S5LsLfg/Twy980iCnYI/AAAAAAAAAIc/NKcfl4QeVj0/clip_image006_thumb%25255B1%25255D.jpg?imgmax=800" border="0" height="376" width="599" /></a></p> <p>For completeness, this is the Seq.sample module extension:</p> <p><a href="http://lh3.ggpht.com/-Cm36iVMMo60/Twy99f017rI/AAAAAAAAAIk/04wgV-paowQ/s1600-h/clip_image007%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image007" alt="clip_image007" src="http://lh5.ggpht.com/-8z65PJkV3qY/Twy9-EBsTWI/AAAAAAAAAIs/hUfT9judcXo/clip_image007_thumb%25255B1%25255D.png?imgmax=800" border="0" height="103" width="521" /></a></p> <p>The nastiest bit of the code in ProcessStream is probably towards the end, where the windowedUnderlyingData is manipulated to ensure that the window only contains 1000 samples. It would be nice to do this in a non-imperative way, using Seq.windowed, but the problem is, is that the sequence we’re operating on is only the result of one buffer read operation, whereas windows etc. should operate over the whole data stream, and the sequences can’t be combined into a single sequence using <i>yield!</i> as they’re all generated asynchronously. Similarly, the buffer takes non-overlapping windows over the input sequence, and without taking a buffer over the whole stream, it may miss samples off the end of the sequence portions. Tomas Petricek has written a library, AsyncSeq, <a href="http://tomasp.net/blog/async-sequences.aspx">http://tomasp.net/blog/async-sequences.aspx</a> which I may investigate in a later post. </p> <p>The alternative to this would be to implement mailbox processors, having different agents for the buffering, windowing and sampling operations. I did start investigating this, but didn’t feel happy with them being pull-based (they don’t return data unless asked). I could have set them up to act more like a dataflow network, but it does seem to go against their intended use of managing state between different threads. I may revisit this in a future blog post.</p> <p>I feel that even though F#’s does have nice features which helped to quickly implement the app, RX would probably be a better fit. I guess I won’t know until I implement it in RX and compare the differences.</p>Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com2tag:blogger.com,1999:blog-848045343259901678.post-68350556635527268102011-10-30T11:49:00.000-07:002012-03-24T03:23:05.541-07:00Family tree timelines<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVT9ZvjxAeinR_j_u0PhKkk-g7kV_e2fQ3rK8g1oo7Sfi9cQ2Le0_C6JkX5kIH6R3IIs4aSoCFTwSKPlotUbpyrCxSVEHzKl-E6faJUr1CCqB1DXUCDBIGnZSRRiGo0oHcW2qJ7zKsVJw/s1600/Timeline.png"><img style="cursor: pointer; width: 663px; height: 352px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVT9ZvjxAeinR_j_u0PhKkk-g7kV_e2fQ3rK8g1oo7Sfi9cQ2Le0_C6JkX5kIH6R3IIs4aSoCFTwSKPlotUbpyrCxSVEHzKl-E6faJUr1CCqB1DXUCDBIGnZSRRiGo0oHcW2qJ7zKsVJw/s400/Timeline.png" alt="" id="BLOGGER_PHOTO_ID_5669359561716190082" border="0" /></a><br /><br />As I talked about in my previous post, I created a simple family tree visualisation program to let me know which of my ancestors I have the most interest for.<br /><br />I also prototyped up a timeline view - in the above image. It's probably pretty obvious that it's a prototype, as there are no labels to identify any of the individuals.<br /><br />The idea for this came about as genesreunited has a non-validated freeform text field for all entries of dates. There's no way to validate whether this information is valid for GEDCOM export - the GEDCOM spec lets various dates, such as approximate dates, bounded dates, dates within a specific quarter etc. to be specified but obviously the date has to be specified in the correct format.<br /><br />I wanted to check that all dates were both in the correct format, and actually valid (i.e. check for nonsensical dates such as parents born after their children etc).<br /><br />The idea behind the colouring is for green to show a GEDCOM format valid date, yellow to specify a missing date (with the date inferred), and red to indicate an invalid date.<br /><br />The opacity is to indicate the 'confidence' in a date - with a specified date range not being at full opacity. Also, the dates can be inferred using a set of rules (e.g. parents should be at least 12 years older than their children, a child is probably born within a year of his or her baptism, the first child is born within a year of his or her parents marriage etc.). These rules could obviously get quite complicated.<br /><br />The layout matches pretty much the layout of the ancestor chart, with ancestors being adjacent to each other. I was fussing over this a little bit - I thought it'd be nice for the descendents on the chart to be nearest to each other on the x-axis), but there's no way for this to happen through the tree.<br /><br />I was feeling pretty happy with this, and thought that it's probably worth putting into the app (with some tweakes such as making the y-axis an adorner layer that adjusts with scale etc), but then I found that there's some software which has a pretty-much identical view to this (I did google around for this before implementing, I obviously didn't google hard enough).<br /><br /><a href="http://progenygenealogy.com/products/timeline-charts/genelines-sample-charts/pedigree.aspx">Progeny Genealogy</a> has an identical layout (OK, turned on its side). It even has the same idea of using opacity to indicate what data is estimated (not varying opacity, but even still). I guess that there's only so many ways to solve a problem, but it's still gutting when you think you've had an original idea!Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0tag:blogger.com,1999:blog-848045343259901678.post-91170966438590555242011-10-29T12:35:00.000-07:002012-02-15T13:13:09.093-08:00Windows 8 Metro Development ExperienceI’ve been running the Windows 8 developer preview for a few weeks, and thought I’d blog about my experiences in converting a Silverlight app I’ve written into a Metro one. I’ll first describe the app, then my experiences developing it.<br /><h3>The App</h3> <p>The application is a family tree viewing application. It came about as I’ve found a few branches my family tree back to the 1600s (with help hooking up with other people on GenesReunited). It isn’t very friendly if you want to see the whole tree as it shuffles all of the individuals around to minimise the screen real estate, which is good for printing, but not so good to make sense of the tree. I was interested in seeing which of my ancestors I had the most history for, and which I wanted to research next, so I created an app to show ancestors without siblings, and to favour clarity over screen real estate.</p> <p>(When I started writing this app, there didn’t seem to be any family tree apps which showed the whole family tree. I’ve since found that <a href="http://gramps-project.org/">Gramps</a> uses GraphViz to do exactly that).</p> <p><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjC_J8qQ8h3uxmzq7qvl-gIwwo8FaLXZ6xhIG8yaiO2TPysmk88rkLCw83ZFyKqt3TTkbOoM86dwc9XqDf3MPr9f8MVvCM80UJp8VJ-Fjv-JBRBYMxprHJ6pyMkam1u-3xdL2D3T5WHWQM/s1600/FullTree.png"><img style="cursor:pointer; cursor:hand;width: 400px; height: 225px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjC_J8qQ8h3uxmzq7qvl-gIwwo8FaLXZ6xhIG8yaiO2TPysmk88rkLCw83ZFyKqt3TTkbOoM86dwc9XqDf3MPr9f8MVvCM80UJp8VJ-Fjv-JBRBYMxprHJ6pyMkam1u-3xdL2D3T5WHWQM/s400/FullTree.png" alt="" id="BLOGGER_PHOTO_ID_5669000264770816962" border="0" /></a></p> <p>The screenshot above shows that Metro apps really don’t much unnecessary chrome. Any infrequently used commands are hidden in the app bar at the bottom of the screen. The next screenshot shows the app at full zoom with the application bar hidden.</p><h3><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj29OwhLj-SSTH7kTfg3FFanE9DUKgxNPlGbl_0c5C32Qc0mjzOFYmq5MeTo7w7DC8_BkC3a88-zVbcxDMygEewaEq8BGAb93XhyphenhyphenjTuR78dmpp2qC9PekOFl-IgrZnVEZnzl2qZwGo_7b4/s1600/Zoomed.png"><img style="cursor:pointer; cursor:hand;width: 400px; height: 225px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj29OwhLj-SSTH7kTfg3FFanE9DUKgxNPlGbl_0c5C32Qc0mjzOFYmq5MeTo7w7DC8_BkC3a88-zVbcxDMygEewaEq8BGAb93XhyphenhyphenjTuR78dmpp2qC9PekOFl-IgrZnVEZnzl2qZwGo_7b4/s400/Zoomed.png" alt="" id="BLOGGER_PHOTO_ID_5669000536216039058" border="0" /></a></h3> <h3>Development</h3> <p>Now I’ve laid out the background, I can talk about the development experience, and some of that will include talking about Windows 8. First off, Metro’s UI is a bit toy-ish, but I can see that it will feel slick on a touchscreen (I’ve got a Windows Phone 7 device, and am struck by the similarities). The green is pretty garish thoughout, but I’ve attempted to follow this look in my app.</p> <h4>UI and controls</h4> <p>It doesn’t feel as if Microsoft has paid enough attention to using Metro apps with the mouse and keyboard yet. A case in point is in the zooming in this application – I’ve provided a slider in the search bar, but it’s unclear whether there will be a system-wide gesture mapping to pinch-to-zoom in the release. This was necessary for testing, as even though Visual Studio has a simulator which can be used to simulate gestures, the zoom gesture uses the mousewheel, which my laptop does not have.</p> <p>The actual conversion was surprisingly simple (they promised that this would be the case on the Build videos), but I’m glad that I didn’t hit any blockers. My ViewModels needed minor changes (which I’ll talk about below), and most of my Xaml just got moved across.</p> <p>The ScrollViewer now allows zooming and panning, so I was able to use this instead of my own ZoomPanControl. Strangely, the zooming is performed by a method instead of a dependency property (making it easier to animate zooms etc). Also, even though it’s drawing visuals, the zoom seems to take a bitmap at full zoom and simply scale that down (this is speculation on my part, but to my eyes the scaled content looks a lot worse than a WPF version which uses scale transforms). Here’s a WPF version of my app at similar zoom:</p> <h3><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihwHRRSTrHwge2FZhyxSYsGVTtYQKk__ZEkcPDPbTfIbJWrGjeRkCz_PxgDIiWN2OFK0lt61z5hEXnj6SPrW79Jf6YNSxTVRawCAiGRrPyPYSrFtllWAwNb-s9NwQaT8NneJLBj3oMXho/s1600/wpf.png"><img style="cursor:pointer; cursor:hand;width: 400px; height: 162px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihwHRRSTrHwge2FZhyxSYsGVTtYQKk__ZEkcPDPbTfIbJWrGjeRkCz_PxgDIiWN2OFK0lt61z5hEXnj6SPrW79Jf6YNSxTVRawCAiGRrPyPYSrFtllWAwNb-s9NwQaT8NneJLBj3oMXho/s400/wpf.png" alt="" id="BLOGGER_PHOTO_ID_5669001007946059458" border="0" /></a></h3> <p>The text is almost readable, and the connecting lines aren’t suffering from the ugly aliasing.</p> <p>There are some further niggles, it seems that the ScrollViewer has a minimum zoom factor of 0.1. Additionally, the slider control, even though it was set with a minimum of 0.05 and maximum of 1, would only display values of 0 and 1 on the popup.</p> <h4>Coding</h4> <p>Now I’ve described some of the structural changes, I can describe some of the coding changes. Most of the changes were pretty trivial in the ViewModel – apart from the use of <strong>async</strong>. The Silverlight of the app was using the Reactive Framework. Other than reading a couple of articles, I hadn’t gotten into looking into C# 5 async yet, but it was pretty trivial to switch over. Testing it is another matter – I ended up downloading the Async CTP to see their unit testing sample, and found that it was the most complex sample throughout. The Reactive Framework allows you to simply control <em>Virtual time </em>using the scheduler, and I haven’t seen anything so simple or elegant for C# 5 async (though testing Observable.FromAsyncPattern methods are similarly tricky to test, as they don't use the scheduler requested; relying on the underlying IO Completion ports for scheduling the work).<br /></p> <p>[Edit] I originally blogged that I was concerned that there wouldn't be a version of the Reactive Framework for .NET 4.5/WinRT, following some forum rumours. However, the guys have a <a href="http://channel9.msdn.com/Shows/Going+Deep/Bart-De-Smet-Rx-Updat-NET-45-Async-WinRT">.NET 4.5 RX</a> build ready.<br /></p> <p>Other changes in the code mapped naturally across – the FileDialog now returns an IInputStream, but this has a method AsString() to map across to a .NET stream. The List class has had a few of its methods (such as the Sort overload taking a delegate) removed, annoyingly.</p> <p>Also, now that asynchronous calls are so pervasive, I’m surprised that the Silverlight Toolkit BusyIndicator didn’t make it in.</p> <h4>Visual Studio</h4> <p>First, the unit testing tool is definitely pre-beta. I didn’t actually finish investigating how to unit test my C# 5 async method conversions, as I couldn’t stomach using the tool any longer. I’m also not overly-enamoured with the new Find dialog. Otherwise, it seems to be pretty stable.</p> <h3>Metro-ising</h3> <p>So far I’ve spoken about how easy it was to convert over a Silverlight application to Metro, but Metro does allow many compelling features to be very easily added to the application. Charms could be provided to allow searching of family tree information, and to allow the images to be easily shared.</p> <p>Once I get hold of a touchscreen device, I’d love to add some snap points to the chart.</p> <h3>Other thoughts</h3> <p>Overall, I’m pleased with the development experience of targeting Windows 8 Metro. I haven’t spoken about the actual platform in this blog, but after the months of silence and confusion about Windows 8, it’s all good news. .NET developers are still first class citizens.</p> <p>I’m happy that WinRT is back to native code, and as I have experience in C++/CLI I’m very happy that C++/CX seems pretty much identical. From a .NET coder point of view, it’s a little concerning that .NET apps will have a slight performance disadvantage from the COM Interop of the WinRT projected types, but I suppose that’s simply the whole ‘use .NET for productivity, C++ for performance’ argument. And having worked on a few apps that had all of the performant (and usually legacy) parts of the system in C++, with a .NET UI, and the subsequent marshalling layer, it’s quite heartening to think that now we can stay in C++ and write a fast and fluid UI without changing languages.</p>Anonymoushttp://www.blogger.com/profile/12874508418428339669noreply@blogger.com0