danushka “silvermace” abeysuriya’s blog

Postfix SMTPD with SASL and AUTHMYSQL

Setting up Postfix can be a bit of a jungle of information, I was left scratching my head wondering why Postfix refused to relay an email to an external address even when provided valid user credentials. The issue seemed to be that Postfix SMTPD wasn’t paying any attention to the /etc/postfix/sasl/smtpd.conf file (which contained the auxprop mysql authentication plugin configuration) and hence was falling back to its default PAM based system.

The fix was the most paintfully simple addition to the Postfix main.cf, highlighted bold. This fix seems to only apply to Ubuntu based environments and means that the smtpd daemon will actually pay attention to the contents of /etc/postfix/sasl/smtpd.conf

smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
smtpd_sasl_application_name = smtpd
broken_sasl_auth_clients = yes

more reading: http://ubuntuforums.org/showthread.php?t=297462

Handy VC2008 Macro

So, if you’ve ever tried to drag a folder from the filesystem into VS Solution Explorerer, sure it’ll recursively add all the files, but it wont mirror the folders in the solution, so you end up with every file placed at the root level of whatever “filter-folder” you dragged it into. This is very annoying when you have a project with say ~1,500 files.

I thought about how I could get around this and have the Solution Explorer folders mirror the filesystem, first obvious option was write a shell script or macro to do it. So I fired up VS macro explorer and lo-and-behold theres a Sample macro which ships with VS called “AddDirAsSlnFolder”.

Looking very promising I checked out the source code, and sure enough it does exactly what I needed. The only thing I had to do to get it working exactly right was add “.svn” to the exclusions list (this is down the bottom of the source file and should be clearly visible next to all the other exclusions in the list)

Enjoy.

Getting FCollada to compile with VC++ 2008

When trying to build from the converted VC2005 solution file, you will encounter quite a few errors regarding mismatching declaration of _vsnprintf.

error C3163: '_vsnprintf': attributes inconsistent with previous declaration

the last post in this thread clarifies how this error arrises in common practice. To get it to compile I commented out these lines:

  • FUtils\Platforms.h:145
  • LibXML\config.h:92
  • LibXML\include\win32config.h:90
  • DLLEntry:40 (this one was just a missing symbol or something deprecated in new the PSDK shipping with VC9, the case statement is empty so commenting it out should have no effect)

I should clearly note that I haven’t tested it (running etc.), I simply got it to build to help save our tools guy a bit of time. I should also note that the Unit Tests (FColladaTest) project builds but returns about 60 linker errors. I just thought I’d post here to save someone a bit of hassle if they should run into this error.

FCollada version was 3.05B Win32

-Danu

fun with numeric_limits min

Just a quick note, I thought others might find this helpful as well. In our level compiler, a bug which somehow went un-noticed for a little while was causing bounding boxes to not be calculated properly. This turned out to be a simple initialization oversight.

Vec3 maxValue, minValue;

ofcourse the default constructor would initialize both vectors to [0,0,0]. This is useless when trying to determine a bounding box which lies entirely in the negative part of all axies (because no vertex in that area will ever excede maxValue, which has incorrectly been initalized to zero). The fix was simple, but it reminded me of another C/C++ quirk over numeric limits.

The fix, one would think, would be something like this:

const float FLOAT_HIGHEST = std::numeric_limits<float>::max();
const float FLOAT_LOWEST = std::numeric_limits<float>::min();
Vec3 maxValue(FLOAT_HIGHEST, FLOAT_HIGHEST, FLOAT_HIGHEST);
Vec3 maxValue(FLOAT_LOWEST, FLOAT_LOWEST, FLOAT_LOWEST);

But, you’de be wrong. If you were using numeric_limits<int> that would work fine, but numeric_limits<float>::min acutally returns the lowest representable positive float (this is to help calculate another issue with IEEE floats see: http://en.wikipedia.org/wiki/Denormal).

Boost provides a clean solution for this in their “bounds” package.

I’m tempted to just go “-numeric_limits<float>::max()” (that is pretty much what the boost code will compile down to) but I like consistency, and I don’t want to worry about any (if any exist?) unmatched upper and lower values like 32 bit signed ints have, I’m no float expert…

“V 48hour film contest”

Myself and a few friends put together a team to write, direct, shoot & produce a short film in 48 hours, May 8th-10th.

Our entry is in, it’s called “Six Minutes” it’s a “Real-time” action movie. We are in the Auckland Heats, and our team (Wise Guys) is “Heat 19″ which will be screening on May 23rd in Auckland, see http://www.48hours.co.nz/

We’re really happy with it, considering it’s all shot on handy-cam and no-one on the team has any professional film making experience what-so-ever haha. But please trust me when I say, it doesn’t really show..

Just thought I’d blog about it. Why? I don’t know, it seems like we achieved something, and I feel like letting people know.

Multi-platform development

Just a short post. Today I took a quick tally of the technologies we’re using. 

  • OpenGL 3.0
  • OpenGL-ES CM 1.1
  • MSVC++ 8
  • GCC 4.0.1 (Apple) & GCC 4.4 (Linux)
  • C# .NET 3.0 (or 2.0?)
  • XNA 3.0
  • XCode 3.1
  • Eclipse CDT 5.1
  • Visual Studio 2005
  • SVN
All of this is spread across 5 operating systems:
  • Apple MacOS 10.5
  • Apple iPhone OS 2.2.1
  • Microsoft Windows XP SP3
  • Microsoft Windows Vista SP1
  • Ubuntu Linux 8.10
On 3 devices:
  • XBox360
  • iPhone
  • PC (Windows and Linux)
All for just this one project. We have a very talented and competent team, and we’re able to work across these platforms quite effectively which really amazes me at times.
If I had to provide a solitary tip, I would put forward that developing a good testing strategy and collaboration infrastructure (for us it’s VoIP/IM and a dedicated server for SVN) save time. alot of it. Oh, and some basic project management software is a must.

You live and learn, C++

I’ve switch jobs and im working with C++, Objective-C++ and alot of cross platform stuff. Which is a nice change of pace from Java.  I’ve extracted out the C++ unit testing framework from one of our IP projects and am actively using it to write unit test for a data exporter im working with, which brings me to my point for this post. And thanks to Kevin who helped me figure this out.

In C++, it turns out function-call arguments are not guaranteed by the standard to be evaluated in the traditional “left-to-right” order. Instead, it’s totaly implementation defined, and I’ve found that on VC++ (2k5) its as you would expect about 90% of the time, but there are certain situations where the compiler decides to mix things up.

the problem was noted in this piece of code:

Vec3 boundsMin( reader.readF32(), reader.readF32(), reader.readF32() )

the compiler translated it into something along these lines:

call readF32  /* stores the result in ecx */
push ecx
fstp dword ptr [esp]
call readF32
push ecx
fstp dword ptr [esp]
call readF32
push ecx
fstp dword ptr [esp]
lea [boundsMin]
call Vec3::Vec3

notice how the store instruction “fstp” always writes to the memory location specified by esp? now the assembly generated for Vec3 constructor’s uses the “rep stos” instruction to “block copy” the contents of the location esp refers to. Initially, all this code looks like it happens how you would expect: “left-to-right” but the reason it doesnt work as expected is because the “push” instruction is decrementing the value in esp by 4 and writing the value of the operand to the top of the stack every time its called.

What does this mean? in simple terms, yes, the reads happen in the order A,B,C but the resulting values are written into the stack-frame effectively backwards C,B,A (I say effectively because the “rep stos” instruction reads from lower memory to higher memory, but C was written to lower memory (because it was written last, hence esp would be at its lowest value), B in higher memory after C and A in even higher memory after B). This is where the non-guaratee by the standard comes into play, the only reason the compiler does this is because there is no requirement to do otherwise. hence all the values for this particular vector were having their x and z values swapped consistently in “Debug” generated code.

Fun times.

Oh and the lesson from all this: don’t inline function argument-expressions which have dependent reads. e.g. reading from streams.

For more reading: http://www.embedded.com/story/OEG20020429S0037
and: http://compilers.iecc.com/comparch/article/95-07-089

Relearning C++

I haven’t blogged in a while now, so I thought I’d write something about a new personal project I’ve started. I entitled the post “relearning C++” because though I know lots of the language quirks and the nitty-gritty of C++, I have considered the past year or so a total re-introduction in the way I apply C++ and in particular, go about developing software using C++. For 10 years now I have used C and C++ in my daily life, being self taught from an early age it has lead to lots of painful (to say the least) learning experiences and I would have to say that up until recently my C++ code has had that “self taught” feeling to it - Most of you know what I mean, mediocre variable names, slightly odd coding styles, choosing brevity over clarity and so forth.

The last few years working with Java in the industry has given me a little bit of insight on producing quality software over software that just does what it’s supposed to, this is the “relearning” I refer to.

The new project has been written in a unit-test centric manner, that’s not to say its TDD nor has it 99% code coverage, but unit-test are part of the center stage. Additionally, we are making good (not over-the-top) use of the collection of powerful Boost libraries, and using these meta-programming/template heavy libraries has definitely learned me a thing or two about C++ - I guess that’s the thing about C++, no matter how long you’ve been using it effectively, there’s just so much to know and there are always those “ahhh-haa [forgot about that!!!]” moments and I guess for me this makes things a bit more challenging, but a little bit more fun (but probably not as productive as it could be).

I wrote the unit testing framework for our project, quite a simple suite/runner system. The system has been valuable for us as a  2-man-decentralised-after-hours-team (2MDAHT) - basically self documentation for code and use-cases and full regression testability, which when communication is not as often as we’d like it, is a great way to make sure we’re always taking two steps forward and none back.

The system is also written thread-safe from the ground up, the unit tests and our design-test-implement-test  philosophy allow us to see deadlock situations during the design-test phase and then during implementation allow us to more easily replicate and fix deadlocks by drastically narrowing the window of code which could be causing the problem.

If you had asked me 2-3 years ago whether unit-testing C++ code was worth it, I would’ve almost certainly would have said “no” or at best “it depends”, today I would go so far as to say it would be a “must” (however I do appreciate the feeling of impending doom when faced with retro-fitting legacy C++ for unit testing).

Weve 2.0.3 Beta

After a long absence, I’ve brought back WEVE (Windows Environment Variable Editor). I use it a lot, and quite a few people got some good use from Weve 1.5. This came about when today I got an email from Robert Hadjiyski asking what ever happened to WEVE, and if I would be able to provide the source code so that he may make bug fixes/improvements.

This reminded me that I had Weve 2.0 sitting around on my HDD gathering dust. So I’ve put the binaries up in a ZIP file, and I have applied to have a source forge page created to host/manage the project’s source code. At some stage I’ll update the MSI installer project and stick that version up too.

Hope people like the 2.0 improvements, alot of handy features that make WEVE 2.0 even easier and more intuitive to use than 1.5.

Get it from the WEVE page

XSLT and Java Quick Tips (Xalan)

Recently at work, I’ve been using XSLT transforms in our backend quite a bit. But the issues I’ve run into took stupid amounts of effort to resolve and it wasnt easy to find any kind of list of do’s and dont’s - so I’m gonna make one from the stuff I’ve just discovered!

  • When dealing with DOMSource and DOMResult - always setNamespaceAwareness(true) on your DocumentBuilder/DocumentBuilderFactory !!! - otherwise you end up with empty Documents and no indicators as to why!
  • Remember that an XSLT transformation is not guaranteed to return an XML document! Hence, Don’t automatically assume a DOMResult( docBuilder.newDocument() ) will return a valid object - if your XSLT doesn’t produce valid XML, you’ll get an empty DOMResult document node!
  • Validation settings affect whitespace awareness - if you don’t have validation on, whitespace is NOT ignored, so make sure your XSLT is formatted correctly - the most common problem area is when you try to prettyfy your .xsl file using indentation etc. Remember that XSL is pretty much an in-place tempate file [Copy Contents -> Replace Stuff exactly where it's asked for -> Dump result!]

Lastly, and easy way to transform input XML and pretty print it.

TransformerFactory transfac = TransformerFactory.newInstance();
Transformer trans = transfac.newTransformer();
trans.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no");
trans.setOutputProperty(OutputKeys.INDENT, "yes");

Document doc = buildXml(); /* get your XML document from somewhere.. */

StringWriter stringWriter = new StringWriter();
StreamResult result = new StreamResult(stringWriter);
DOMSource source = new DOMSource(doc);
trans.transform(source, result);
System.out.println(stringWriter.toString());