Yet another Hacker News

56 points theanonymousone 3 days ago 13 comments

austin-cheney 3 hours ago | parent

The \s character class in JavaScript regular expressions includes 25 characters. I don’t know if that perfectly matches the Unicode list of 25 characters, but it does make identifying white space simplistic.

White space characters are challenging to differentiate in human readable text, but JavaScript makes that perfectly clear to humans by splitting a string on every character, which is helpful for debugging.

_def 1 hour ago | parent

The article covers that:

    The odd one out above is NEXT LINE (U+0085). Unicode explicitly includes it in White_Space, and Python accepts it too, but JavaScript doesn't. Instead, JavaScript recognises ZERO WIDTH NO-BREAK SPACE (U+FEFF), which Unicode doesn't include in the set.

adornKey 3 hours ago | parent

With font.color = white even an ordinary "W" could count as whitespace... Something to remember if you see whitespace with an extraordinary width out there in the wild...

LoganDark 2 hours ago | parent

Whitespace depends on the context. For example, you could have characters that are not counted as whitespace in Unicode, but end up rendering as whitespace when displayed in a particular context. The only real way to check for this is to actually try to render it and see what happens. There are some other ways to approximate it, like checking the font that you use for rendering, or the bounds of the glyph, or other things like that, but the only way to really know for sure in all cases is to just render it and see what happens.

tjoff 1 hour ago | parent

My tab renders as: " > ". I still consider it whitespace.

What turns into whitespace due to rendering? Just because my font lacks that symbol doesn't mean it is whitespace.

LoganDark 48 minutes ago | parent

One example is a field that clips its contents to contain zalgo, but inadvertantly to the point where some glyphs can render completely out of bounds. Sometimes the clipping is so egregious that even descenders of normal letters like 'g' get cut off. (This is more common than you'd think.)

tjoff 37 minutes ago | parent

I get the desire to filter that out, not sure whether I'd classify it as whitespace though.

efilife 2 hours ago | parent

This code is very simple but I still don't really get it. Why is the hashing needed and how is it faster than a regex? Can someone smarter than me explain?

usrnm 1 hour ago | parent

No branching (if statements), no heavy CPU operations, just a multiplication by a constant, a shift and a lookup in a small table that will get cached fast

SerCe 1 hour ago | parent

You need a hashtable to check whether this character is part of a given set. And the perfect hashing-based implementation is the most efficient form of a hashtable since you only need a lookup and an equality check. I included a small visualisation in the article that illustrates how it works, so if you try clicking on the elements, it will show you how the calculations are made.

lelanthran 23 minutes ago | parent

> Why is the hashing needed and how is it faster than a regex? Can someone smarter than me explain?

It's a perfect hash (i.e. no collisions) that hash to an entry in a 32-element array.

Very similar to how we usually do quick lookups of strings for `switch()` statements: https://github.com/lelanthran/libxtree/blob/878cb4fc11ffd617...

smallstepforman 1 hour ago | parent

With bitmasks, if they just assumed bit 5 (whatever) as zero=whitespace, they would have saved billions of cpu cycles per device per day. With 32 bit unicode, that gives you 32 unique whitespace chars.

Should have got guys that grew up with the 8-bit micros to write up a spec.

pixelbeat__ 1 hour ago | parent

I think horizontal white spacing should be distinguished, as that's what most care about with splitting etc.

Have a look at how GNU coreutils classifies each type of spacing (in a cross platform consistent way) at https://github.com/coreutils/coreutils/blob/master/src%2Fsys...