Changes to the Swift Standard Library in 1.1 beta 2

Less than a week after the last beta was released, Swift 1.1 beta 2 is up. And yes, it’s officially Swift 1.1, as displayed by running swift -v from the command line.

Presumably we’re gearing up for a new GM alongside Yosemite, as there are almost no changes to the standard library in this release, much like when the GM was almost upon us for 6.0. Not surprising, since the Swift team had already confirmed on the dev forums that 6.1 was going to be a fairly small release.

The only material change are to the RawRepresentable protocol, and the _RawOptionSetType which inherits from it:

  • RawRepresentable’s Raw typealias is now RawValue.
  • Instead of a toRaw function, it now has a read-only rawValue property.
  • Instead of a fromRaw class function, it now defines an init? that takes a raw value.
  • _RawOptionSetType also now defines an init that takes a raw value, rather than a class function fromMask.

This matches the release notes, which state that enums construction from raw values have been changed to take advantage of the new failable initializers introduced in the last beta.

Interesting thing to note: while RawRepresentable has an init?(rawValue: RawValue) method, _RawOptionSetType has an init(rawValue: RawValue) method. This might seem odd at first – _RawOptionSetType inherits from RawRepresentable, shouldn’t they have the same kind of init? But since init is more restrictive than init?, it works, as an optional type can always be substituted with its non-optional equivalent. _RawOptionSetType is essentially restating RawRepresentable’s raw initializer as non-optional after all.

Here’s looking forward to a Swift 1.1 GM, and maybe after that on to 1.2?

Accidentally putting a loop in your loop

Joel Spolsky wrote a good article back in 2001 about knowing when the function you’re calling has linear complexity,1 and not accidentally putting a loop in your loop,2 getting quadratic complexity without realizing it. He was talking about C string algorithms like strlen and strcat, but it applies just as much to several Swift standard library functions.

When a function only has versions that take a sequence, like the seductively-useful contains, equal, min– and maxElement, it’s linear.3 Some depend on their input: countElements can be done in constant time if you give it a collection with a random-access index, but linear if not. Same with distance and advance index operations, which is why extending String to support random-access indexing using them is probably a bad idea. Be careful, not everything that could be optimized may have been. For example, ClosedInterval.contains is presumably O(1) but there doesn’t appear to be an optimized overload for the non-member contains, which only has versions that take a sequence.

The Swift team have helpfully put the complexity of certain functions in the documentation. Array’s reserveCapacity, insert, and replaceRange are O(N).

And so is… removeAtIndex. In a previous article, I mentioned the following code to remove all occurrences of a given value from an array has an efficiency problem:

func remove
    <C: RangeReplaceableCollectionType,
     E: Equatable
     where C.Generator.Element == E>
    (inout collection: C, value: E) {
        var idx = collection.startIndex
        while idx != collection.endIndex {
            if collection[idx] == value {
            else {

The documentation for removeAtIndex says: 4

Remove and return the element at the given index. Worst case complexity:
O(N). Requires: index < count

That is, the worst-case time it takes to remove an element increases in linear proportion to the length of the collection. That's not surprising – if you remove an element of an array, you have to shuffle each element after it down one. The larger the collection the more things to shuffle. Maybe your element is near the end, maybe your collection is a linked list that can remove elements in O(1), maybe it’s magic and has all sorts of cool optimizations – hence O(N) is the worst case. But still, it’s probably not best to call it within another loop.

Here, it shouldn't be necessary. We're already iterating over the collection, so ought to be able to combine the deletion and shuffling down together as one operation. Here's a new version of remove that does this, modelled on the C++ STL equivalent:

(incidentally, it also removes the need for questionable assumptions about how indexes behave when you remove elements, the subject of the previous article)

func remove
    <C: protocol<RangeReplaceableCollectionType,
     E: Equatable
     where C.Generator.Element == E>
    (inout col: C, value: E) {
        // find the first entry to remove
        if var advance = find(col, value) {
            // advance points to next element to test,
            // rear points to where to copy it to
            // if it's a keeper
            var rear = advance++
            while advance != col.endIndex {
                if col[advance] != value {
                    col[rear] = col[advance]

This version breaks the removal into multiple steps. First, it uses find to locate the first entry to remove (and if it doesn’t find one, does nothing more). Next, one by one it moves the subsequent elements down on top of that entry. When it encounters more entries to remove, it skips over them (i.e. it increments advance, the index of entries to examine and maybe copy, but not rear, the index of where to copy to, and they aren’t copied).

Finally, when all this is done, the collection should have all the non-removed entries at the front, and some meaningless garbage at the end. This end section is the length of the number of removed entries (though it doesn‘t contain the removed entries – entries were copied, not swapped). This is trimmed off by a call to removeRange, which is also O(N). But that’s fine – two O(N) algorithms in series is still O(N).

By the way, in this last step it differs from the C++ STL version which, in a quality bit of user-unfriendliness, requires the caller to do the final remove step, instead leaving the collection with the garbage still at the end. There‘s good reasons for this (because iterators), but it’s a nasty gotcha for newbies. I’d chalk this one up as a win for Swift’s approach to generic collection algorithms. 5

Note that for this to work, the collection also needs to support the MutableCollectionType protocol, because that’s where the assignable version of subscript lives for some reason.6 In fact, that’s all MutableCollectionType adds. By the way, this whole optimization is based on the assumption that the assignment version of subscript runs in constant time. It should do, right? Copying a value into a position in an array shouldn’t affect the rest of the array, so it shouldn’t matter how long it is.

A quick test with a reasonably large array shows that this new version of remove does indeed run quicker (and more consistently) than our first version. Yay.

Then you try and use it on a string, and your celebration is short lived. String doesn’t implement MutableCollectionType. Huh, what’s that about?

What this doesn‘t mean is that String is immutable. It’s totally mutable. Mutating is what RangeReplaceableCollectionType is all about. This is a chance to make an important if maybe obvious point: just because your function takes an object via a protocol that doesn’t allow mutation doesn‘t mean that object is immutable. It just means you can‘t mutate it in your function.

My guess for why strings don‘t support MutableCollectionType? Because that assumption above, about subscript assignment being O(1), doesn’t hold. Remember, individual elements of Swift strings are of variable length. What if you replaced a longer character with a shorter one? We’d be back to square one, having to shovel all the subsequent characters down to fill in the gap. Worse, what if you replaced a shorter entry with a longer one? The string would get bigger, maybe even need relocating to a newly allocated chunk of memory.

This would be another example of signalling more from protocols than just what functions are supported. They can tell you about fundamental properties of the object. Perhaps MutableCollectionType is like MutableInConstantTimeCollectionType. Course, on the other hand, I could be reading waaay too much into String not implementing it. It could just be an oversight – only time (or one of the Swift devs) will tell.

String does support replaceRange as an alternative to subscript assign. But its complexity is, you guessed it, O(N). And after crowbarring it into the second algorithm above, tests suggest it’s no faster than the removeAtIndex version.7 So if you really have a burning desire to remove some characters from a huge string, maybe find another way. Perhaps you can trade some space for that time.

  1. That wikipedia entry on complexity, like so many wikipedia entries on mathematical topics, is pretty beginner-unfriendly. If anyone has a good beginner’s guide link I could replace it with, let me know. An (admittedly cursory) google search doesn’t turn much up. 
  2. I’d have called this post “Yo, dawg” but then I’ve already done that once. 
  3. Unless it does something like return the first element in the sequence, obvs. Hmmm, maybe I should be less paranoid about people finding errors in my posts. 
  4. Actually it doesn’t, not on the protocol version anyway. The stand-alone function version (that takes that same protocol) does though, so I’m taking the liberty of giving that instead. 
  5. It‘s not all wins, mind. The STL’s iterator-based functions are much easier to use with subranges of collections. Don‘t get me started on slices… 
  6. Wait for it… 
  7. though this is possibly due to my lack of deftness with a crowbar. 

Changes to the Swift Standard Library in Xcode 6.1

Congratulations to the Swift team on releasing a 1.0 GM! Unsurprisingly, there were no changes that I could spot1 between beta 7 and the GM standard libraries.

But as they said in their blog post, “Swift will continue to advance with new features, improved performance, and refined syntax. In fact, you can expect a few improvements to come in Xcode 6.1 in time for the Yosemite launch.” As such, the first beta of Xcode 6.12 saw more changes to the standard library than we saw in the pre-1.0 beta, and here they are:

  • As flagged in the release notes, all the integer types have acquired truncatingBitPattern initializers for each of their larger counterparts.
  • The functions join, reduce, sort and sorted have acquired descriptions in the various places they’re implemented.
  • AssertString and StaticString are now Printable and DebugPrintable
  • HeapBufferStorage no longer inherits from HeapBufferStorageBase, which is gone.
  • The Process instance is now declared with let rather than var
  • StrideThrough or StrideTo are now Reflectable
  • StaticString now has a utf8Start instead of start (which still returns an UnsafePointer). It also has a withUTF8Buffer method for using the underlying raw buffer, a unicodeScalar get property.
  • String has lost its compare(other: String.UnicodeScalarView) method.
  • String also now implements several methods extending StringInterpolationConvertible – for a variety of different built-in types.
  • Second versions of assertionFailure and fatalError now take a generic AssertStringType instead of StaticString.
  • equal and startsWith‘s predicate functions have been renamed isEquivalent.
  • maxElement and minElement‘s input parameter has been renamed elements.
  • The toString function now says it returns the result of printing, not debugPrinting, an object.

There’s a new unsafeAddressOf function that takes an AnyObject. Apparently there’s “not much you can do with this other than use it to identify the object”.

There is a new protocol, UnicodeScalarLiteralConvertible, that follows the now-familiar literal converter pattern: a ThingType typealias, a convertFromThing class function, and a library-level typealias for ThingType for the standard type for these literals to convert into, in this case a String.

The ExtendedGraphemeClusterLiteralConvertible protocol now inherits this protocol, so the objects that implement it (Character, String, AssertString and StaticString) also now implement convertFromUnicodeScalarLiteral. And the UnicodeScalar struct now implements UnicodeScalarLiteralConvertible instead of ExtendedGraphemeClusterLiteralConvertible.

Here’s looking forward to more enhancements as Swift continues to develop.

  1. Unless there’s one hiding in those pesky re-ordered operator overloads… 
  2. Does this mean this is Swift 1.1? 

Changes in the Swift Standard Library in Beta 7

There aren’t any.

Well, actually there’s one:

  • The ImplicitlyUnwrappedOptional type, which ceased to be of BooleanType in beta 6, no longer has the associated vestigial boolValue property.

Ok, Ok there are a couple more:

  • The comparison function taken by non-member sort, sorted and partition is now labelled isOrderedBefore (matching the member versions). Not that it’s an external parameter, so no code change needed.
  • Further enhancements to some function documentation comments (for example min() now has a description).

I think… that’s it. I have to admit, I don’t diff all the operators each time, as they seem to constantly reorder with each beta. That’s how I missed that, between beta 3 and beta 4, you stopped being able to use === to determine if two arrays referenced the same underlying storage. Shame on me!

There’s one operator that hasn’t changed. Despite what the release notes say, you still seem to be able to use + to concatenate Characters:

let c1: Character = "a"
let c2: Character = "b"
let s = c1 + c2
// s is now the String "ab"

…but presumably it’s not long for this world. This is in keeping with the trend of having + only work between two collection types, not between collection types and their contained types.

I guess this stabilization of the Swift library is in readyness for the Swift 1.0 release, which is pretty great news. The interesting question is, how will the library keep evolving post-1.0?

Protocols and Assumptions

edit: subsequent to this article being written, the Swift standard library has been updated, and documentation-comments above the relevant methods of RangeReplaceableCollectionType now explicitly state: “Invalidates all indices with respect to self.” Bear this in mind as you read on:

What does it mean to implement a protocol? Possibly more than supporting methods with specific names.

Obviously the methods ought to do what their names imply – isEmpty shouldn’t fire the torpedoes. But as well as basic functionality, there are also guarantees about things like the method’s complexity, and possibly wider implications for how the class itself behaves.

More importantly, protocols might not guarantee a behaviour you think they do and are relying on. Using protocols with generics can sometimes give you the illusion of more guarantees than you actually have.

Suppose you want to write a function remove that removes entries from a collection in-place – that is, similar to the sort function, it takes a collection as an inout parameter, and removes from it all the entries that match a given value. 1

Unlike sort, which just requires MutableCollection, remove would need to use the new-in-beta6 RangeReplaceableCollectionType, which includes a removeAtIndex method. Armed with this, you might write the following: 2

func remove
    <C: RangeReplaceableCollectionType,
     E: Equatable
     where C.Generator.Element == E>
    (inout collection: C, value: E) {
        var idx = collection.startIndex
        while idx != collection.endIndex {
            if collection[idx] == value {
            else {

Embedded in this code are a lot of assumptions about how collections and their indices behave when you mutate them, and some of these might not be valid, depending on the collection type. It works with String, the only explicitly range-replaceable collection in the standard library currently, as well as the secretly range-replaceable Array, ContiguousArray and Slice. But you could easily implement a new type of collection that was range-replaceable for which the above code would explode in flames.

The biggest assumption is that removing an element from a collection does not completely invalidate an index. That is, after you call collection.removeAtIndex(idx), idx remains a legitimate index into the collection rather than just becoming junk.

Next, there’s the assumption that when you remove a entry at an index, that index will now point to the next element in the collection. That’s why, after removing the entry, the code above just goes straight back around the loop without incrementing idx. You could put it another way – when you remove an element, the next element “moves down” to the position indexed by the removed element.

Finally, there’s the assumption that if the element that idx points to is the last element, then what idx will point to after you remove it will be endIndex. Or, to put it another way, as you remove the last element, endIndex “moves down” to equal the index of the last element.

By the way, this last assumption is why the code uses while idx != collection.endIndex rather than the more-elegant for idx in indices(collection). indices returns a Range object between the collection’s start and end, but it would be created before we start looping. Because endIndex is a moving target as we remove some entries from the collection, it won’t work for our purposes. A cleverer version of indices that returned something more dynamic might help, but that could have different undesirable side-effects.

Are all these assumptions legit? Well you can see they obviously are for the Array types, because these just use an integer for their index, with endIndex equal to count. When elements are removed, the rest of the array shuffles down. Even if that resulted in a full reallocation of the array’s memory, the assumptions would still hold because all the index does is represent a relative offset from the start of the array, not point to a specific location.

Strings are trickier, because their index type is opaque. Chances are it’s still an offset from the start of the string, but because Swift characters aren’t of uniform width, 3 that offset doesn’t necessarily increment by a constant amount each time. Still, if this is the implementation, the above assumptions would hold, and experimentation suggests they do.

What kind of collection might not adhere to these assumptions? Well, a very simple doubly-linked list implemention might not. 4 If the index for a linked list were a pointer to each node, then removing that node could leave the index pointing at a removed entry. You couldn’t just loop around without first pointing to the next node in the list:

func remove
    <C: RangeReplaceableCollectionType,
     E: Equatable
     where C.Generator.Element == E>
    (inout collection: C, element: E) {
        var idx = collection.startIndex
        while idx != collection.endIndex {
            if collection[idx] == element {
              // first grab the index of the next element
              let next = idx.successor()
              // then remove this one
              // and repoint
              idx = next
            else {

But then this algorithm would no longer work correctly with arrays and strings! 5

So what’s the solution – should RangeReplaceableCollectionType mandate the kind of index validity behaviour our remove algorithm relies on? Or are the assumptions invalid and we need a better algorithm? (of which more in a later article) The Swift standard library is still evolving rapidly with each beta so it’s possibly a little early to tell. For now, be careful about the assumptions you make – just because all the current implementations of a particular protocol work a certain way doesn’t mean other implementations will.

  1. As opposed to a version that returned a copy, which would be called removed. I thought I didn’t like this convention at first, but I’m warming to it. 
  2. This code has an efficiency deficiency, which we’ll talk about in a later article. 
  3. For an in-depth explanation of Swift strings, see Ole Begemann’s article
  4. Singly-linked lists couldn’t implement removeAtIndex easily, and would probably have some kind of removeAfterIndex operation instead. 
  5. The C++ STL resolves this by having erase return an iterator for the entry just after the erased elements – along with a fairly draconian assertion that any removal invalidates all other iterators (not just ones at and beyond the removed value). But Swift’s removeAtIndex currently returns the removed value, rather than a new index. 

filter, String and ExtensibleCollectionType

String was extended in beta 6 to implement RangeReplaceableCollectionType. This means that, via inheritance, it also implements ExtensibleCollectionType.1

ExtensibleCollectionType is interesting, because it requires the collection to support an empty initializer. This means that, without having to resort to shenanigans, you can write a generic function that takes an ExtensibleCollectionType and returns a new one.

Since they were changed to return eagerly-evaluated results, the non-member filter and map have returned arrays, no matter what. This is a bit frustrating when working with some non-array types, such as String:2

let vowels = "eaoiu"
let isConsonant = { !contains(vowels, $0) }
let s = "hello, i must be going"
// filtered will be an array
let filtered = filter(s, isConsonant)
// and then we have to turn it back into a string
let only_consonants = String(seq: filtered)
// only_consonants is "hll,  mst b gng"

It would be nice to have a version of filter that took a String and returned a String instead of an Array. 3 Even better, it would be nice to have a single generic version that worked on both arrays and strings.

Here’s one:

func my_filter
  <C: ExtensibleCollectionType>
  (source: C, includeElement: (C.Generator.Element)->Bool)
  -> C {
    // use the `init()` from `ExtensibleCollectionType`
    var result = C()
    for element in source {
        if(includeElement(element)) {
            // append is also part of `ExtensibleCollectionType`
    return result

// my_filter returns a String when passed one:
let only_consonants = my_filter(s, isConsonant)

Since this is possible, should Swift’s filter and map be changed to be like this? Maybe, but I can think of a couple of reasons why not.

First, it’d be a bit inconsistent and possibly surprising. Not all collections are extensible collections. Dictionary isn’t. Range and StrideTo even less so – they’re like “virtual” collections that don’t really have individual elements at all. So there’d still need to be versions that took these collections and returned an array. So when calling filter, you’d need to know whether your collection was extensible to know whether you were going to get back the same collection type or an array.

There’s precedent for this kind of thing. lazy gives you back different types depending on what you pass in. But lazy is very explicit. map and filter would be a bit more subtle, and bear in mind subtle maybe-unexpected behaviour was probably the reason lazy evaluation was moved into the lazy family in the first place.

Second, maybe you do want an array back. This can be catered for – declare a second version of my_filter like so: 4

func my_filter
    <C1: ExtensibleCollectionType, C2: ExtensibleCollectionType
    where C1.Generator.Element == C2.Generator.Element>
    (source: C1, includeElement: (C1.Generator.Element)->Bool)
    -> C2 {
        var result = C2()
        for element in source {
            if(includeElement(element)) {
        return result

// same-type version will be used by default
let consonant_string = my_filter(s, isConsonant)
// but if you declare the result as a specific type, the
// second version will be used:
let consonant_array = my_filter(s, isConsonant) as Array

Third, there’s the big gotcha that means this wouldn’t be a good idea, but that I haven’t thought of. If you have, leave a comment or tweet me.

  1. or rather, _ExtensibleCollectionType, which contains the goods, and that ExtensibleCollectionType just inherits without any additions. I’m not sure why it’s done this way, though I’m guessing it’s not for no reason. 
  2. This is of course a horrible piece of code, ignoring upper-case characters, not to mention accented characters, but let’s keep the examples simple. 
  3. Using lazy(s).filter is probably more efficient, since it won’t require the construction of temporary Array. But the issue of it being a two-step process remains. 
  4. Having written the second version, you should probably implement the first one in terms of the second to avoid code duplication. 

Changes in the Swift Standard Library in Beta 6

I managed to catch a copy of beta 6 before it was pulled. Though not a copy of the release notes, so apologies if I duplicate some items (and hopefully don’t misspeak about stuff better explained in them!). On the assumption the binaries will be the same except re-signed, here’s a rundown of the changes to the standard library. (edit: they were)

Feels like Swift might be approaching the 1.0 home-stretch, with the focus moving to stability and Objective-C API interfacing. Nevertheless, plenty of changes to the Swift standard library in beta 6.

By far the largest swathe of changes are additional comments on existing types and functions, which are definitely worth a read and clarify several things. For example, a comment above Comparable makes it clear you only need to define < to be comparable, despite Comparable defining the comparators that aren't <.

Some small bits and pieces:

  • The ?? operator has been updated to include a version to explicitly handle both the LHS and RHS being of the same optional type. I've updated my post with a comment, but it's still worth reading as a case study if you're writing a similar function.
  • Array now has an init that takes a _CocoaArrayType, as well as a noCopy flag, only to be set if the source array cannot be further mutated.
  • AutoreleasingUnsafeMutablePointer is no longer a BooleanType, so no longer has a boolValue property
  • Bit.Zero and Bit.One are now capitalized (don't say we don't pay attention to detail here!)
  • The Bool constructor, which previously took a parameter of BooleanType (which worked because BooleanType has no associated type requirements unlike, say, IntegerType), now takes a generic parameter T that must be of BooleanType. Interesting question to ponder is how this changes the function.
  • COpaquePointer has new constructors from raw memory addresses (these are described as “fundamentally unsafe”, you have been warned)
  • Character is now Comparable
  • The value properties of Float, Float80 and Double (which were of Builtin.FPIEEExx) are gone.
  • The FloatingPointType protocol now includes constructors from all the built-in integer types.
  • ImplicitlyUnwrappedOptional no longer conforms to BooleanType, though it still has its boolValue property (I should avoid using that if I were you).
  • Optional no longer has a hasValue property. You should just use != nil
  • RawOptionSetType no longer implements BooleanType and Equatable but instead implements BitwiseOperationsType
  • The FIXMEs about how StrideThrough and StrideTo should be collections not sequences are gone. They're still sequences.
  • UnicodeScalarView is now reflectable.
  • String has a new extend method that takes another string. This is in addition to the existing extend that takes a sequence of characters.
  • It also has an append function that takes a UnicodeScalar.
  • String is also now Comparable.
  • Strings unicodeScalars property is now writeable.
  • UnicodeScalar now has an init for UInt16 and UInt8 in addition to UInt32
  • UnsafeMutableBufferPointer has several changes. First, it is heavily commented. It's now a RandomAccessIndexType (so you can calculate distances between them, and advance them). And its constructors have been changed to be consistent with COpaquePointer.
  • _ExtensibleCollectionType has added an append function that appends a single element (all the implementors already support this)
  • _RawOptionSetType (and thus RawOptionSetType) is now Equatable.
  • contains now has a version that takes an equatable element rather than a predicate.
  • sorted now takes any sequence, which is way less restrictive as before it required a mutable random-access collection.
  • startsWith now has a version that takes a comparison predicate (but like equal requires both sequences to contain the same type, even if the predicate could handle two different types).

transcode, which I think officially has the longest function signature in the whole Swift library, got a teeny bit shorter as it now returns a (still a bit odd-looking) 1-tuple containing a Bool, rather than a 1-tuple containing a Bool labelled hadError. Oh, by the way, you're not allowed to return 1-tuples with labels as of beta 6.

There is a new AssertString type, which has a lower precedence for overloading purposes than StaticString. How this is achieved is interesting, and a demonstration of how there are still inheritance hierarchies with structs: AssertString implements a new AssertStringType protocol. StaticString also now implements a new protocol, StaticStringType. StaticStringType inherits from AssertStringType. This means StaticString is more specific than AssertString and will therefore “win” in choices for which overload to pick (in the same way a function taking a CollectionType wins over SequenceType if an object supports it, or RandomAccessIndexType wins over ForwardIndexType). The protocols StaticString previously implemented have moved to AssertStringType

There is a family of a new kind of assertion function, precondition. The signatures are very similar to that of assert. The comments suggest precondition is a little stronger – they will still stop program execution even if assertions are turned off. Only with -Ounchecked will they not check the condition. There's also a @noreturn preconditionFailure function that doesn't check anything, just stops execution immediately.

Various new comments in the Swift library suggest use of these new preconditions. For example, a comment above suggests calling preconditionFailure if called a second time after nil has already been returned.

There's a new protocol, RangeReplaceableCollectionType, that defines several new operations on collections such as removing or replacing ranges, as well as insert (insert an element into the middle of a collection), and splice (insert a collection into the middle of the collection).

String implements this new protocol. Interestingly, Array (alongside ContiguousArray and Slice) does not appear to, though it does support all the methods (including new splice and removeRange functions) and if you write a generic function that takes a RangeReplaceableCollectionType, you can pass an Array into it. Declared somewhere more private I guess? Unless I'm missing a bit of indirection somewhere. If you spot it, let me know on twitter.

Following a familiar pattern, several of these new functions, such as slice and the remove operations, are also availabe as non-member functions as well.

Finally, _BridgedToObjectiveCType and _ConditionallyBridgedToObjectiveCType appear to have coalesced into _ObjectiveCBridgeable, but as ever I'll steer clear of discussing bridging topics.

Implicitly converting functions to return optionals

One final post on upconversion of non-optional values to optional. As we saw previously, if you pass a non-optional value to a function argument that expects an optional, it will get automatically converted into an optional by the compiler.

Ken Ferry points out something that goes one step further. If a function takes an argument of a function that returns an optional, and you pass into it a function that does not return an optional, the function will automatically be converted to return an optional.

That is:

func foo()->Int {
    return 1

func bar(fun: ()->Int?) {

// Even though foo returns an Int, 
// you can pass it to bar.
// Inside bar it will return an Int?,
// so this prints "Optional(1)"

What’s more, you can even store or return the converted function:

// alter bar to return the passed-in function
func bar(fun: ()->Int?)->()->Int? {
    return fun

var i = 0
let f = { ++i }

f() // returns 1
f() // returns 2

let g = bar(f)

g() // returns {Some 3}
g() // returns {Some 4}
f() // returns 5

This feature makes sense when you think about it in the context of other implicit conversions – if you can pass non-optional values in to functions and have them implicitly converted to optionals, the logical next step is to be able to do the same for the return value of functions you pass in as parameters.

This only works on the return value of the function, though. The following won’t compile:

func foo(i: Int)->Int {
    return i

func bar(fun: (Int?)->Int?) {

// won't compile - no implicit conversion
// of foo's argument to an Int?

This is because foo is expecting a non-nil argument – it wouldn’t know how handle nil if it received one. However, what would compile is if foo could handle an optional, but bar was expecting a function with a non-optional argument:

func foo(i: Int?)->Int {
    return i ?? 0

func bar(fun: (Int)->Int?) {

// will compile - implicit conversion
// of foo's argument to an Int

This works because any call by bar of foo could convert the non-optional parameter it passes in to an optional implicitly. (thanks to @westacular and @jvasileff for pointing this out)

Just like you can think of passing non-optionals into optional arguments as the compiler automatically wrapping your values in an optional, you can think of the compiler silently wrapping your non-optional-returning function in a closure that calls your function and returns its value wrapped in an optional:

func foo()->Int {
    return 1

func bar(fun: ()->Int?) {

// to do an explicit equivalent 
// of the compiler's conversion:
bar( { Optional(foo()) } )

In practice, the compiler probably does something a bit more low-level. If you wrote the above code, and then stepped through it with the debugger, and you put a breakpoint inside foo, you’d see an extra entry in the stack trace between foo and bar showing the in-between closure. But if you leave the compiler to do it implicitly, you’ll see no such entry.

If you’re interested in digging a bit further into exactly where the compiler is doing the conversion, you can use a feature of the Swift compiler to dump the syntax tree. Here is a dump from the first piece of code in this article:1

% xcrun swiftc -dump-ast main.swift
  (func_decl "foo()" type='() -> Int' access=internal
      (pattern_tuple type='()'))
        (component id='Int' bind=type)))
        (call_expr implicit type='Int' location=main.swift:2:12 range=[main.swift:2:12 - line:2:12]
          (dot_syntax_call_expr type='(Int2048) -> Int' location=main.swift:2:12 range=[main.swift:2:12 - line:2:12]
            (declref_expr implicit type='Int.Type -> (Int2048) -> Int' location=main.swift:2:12 range=[main.swift:2:12 - line:2:12] decl=Swift.(file).Int._convertFromBuiltinIntegerLiteral specialized=no)
            (type_expr implicit type='Int.Type' location=main.swift:2:12 range=[main.swift:2:12 - line:2:12] typerepr='<<IMPLICIT>>'))
          (integer_literal_expr type='Int2048' location=main.swift:2:12 range=[main.swift:2:12 - line:2:12] value=1)))))
  (func_decl "bar(_:)" type='(() -> Int?) -> ()' access=internal
      (pattern_tuple type='(fun: () -> Int?)'
        (pattern_typed type='() -> Int?'
          (pattern_named type='() -> Int?' 'fun')
      (call_expr type='()' location=main.swift:6:5 range=[main.swift:6:5 - line:6:18]
        (declref_expr type='(Int?) -> ()' location=main.swift:6:5 range=[main.swift:6:5 - line:6:5] decl=Swift.(file).println [with T=Int?] specialized=no)
        (paren_expr type='(Int?)' location=main.swift:6:13 range=[main.swift:6:12 - line:6:18]
          (call_expr type='Int?' location=main.swift:6:13 range=[main.swift:6:13 - line:6:17]
            (declref_expr type='() -> Int?' location=main.swift:6:13 range=[main.swift:6:13 - line:6:13] decl=main.(file).func specialized=no)
            (tuple_expr type='()' location=main.swift:6:16 range=[main.swift:6:16 - line:6:17]))))))
      (call_expr type='()' location=main.swift:9:1 range=[main.swift:9:1 - line:9:8]
        (declref_expr type='(() -> Int?) -> ()' location=main.swift:9:1 range=[main.swift:9:1 - line:9:1] decl=main.(file).bar@main.swift:5:6 specialized=no)
        (paren_expr type='(() -> Int?)' location=main.swift:9:5 range=[main.swift:9:4 - line:9:8]
          (function_conversion_expr implicit type='() -> Int?' location=main.swift:9:5 range=[main.swift:9:5 - line:9:5]
            (declref_expr type='() -> Int' location=main.swift:9:5 range=[main.swift:9:5 - line:9:5] decl=main.(file).foo@main.swift:1:6 specialized=no))))))

This is fairly simple to tie back to the original code. We have three sections under source_file: two func_decls, one for foo and one for bar, followed by a top_level_code_decl showing the call to bar passing in foo. And we see that the compiler inserts a function_conversion_expr implicit type='() ->Int?' (line 36).

Incidentally, if you run -dump-ast on a call that requires a conversion from regular type like Int to an optional, you’d see inject_into_optional implicit type='Int?'. And for an implicit conversion to an Any type, you’d see erasure_expr implicit type='Any'.

  1. swiftc dumps to stderr. If you want to pipe longer examples through less, and like me you need to trial-and-error where the ampersand goes every damn time despite how many thousand times you might have typed it before, it’s 2>&1 

Bachman Ternary Overdrive

(ok I think I’ve pushed the puns too far with that headline)

David Owens rightly called me out for pulling a bit of a fast one at the start of my last article on how ?: and ?? differ in behaviour. Not that they don’t, but I was acting as if what the ternary operator version was doing was something perfectly sensible, and that ?? was the funny one. But I glossed over what the ternary operator itself was doing.

Look at this code again:

let i: Int? = nil
let j: Int? = 5
let k: Int? = 6

// this returns {Some 5}
let x = i != nil ? i! : j
// this returns {Some 6}
let y = k != nil ? k! : j

What are the types of x and y? Int?, right? And how is that decided? From the type of the ternary expression. But which part? If you look at the second and third part of the expression, they’re different types. i! and k! are of type Int, whereas j is of type Int?.

That’s no good! Both possible values from the ternary expression have to be able to become the same type, because the type of x is determined at compile time, before you know what i or k contains.

For example, this won’t compile:

let s = "hello"
let i = 123
// x can't be either a String or an Int
let x = i > 0 ? s : i

But note I said become the same type, not be the same type. As we saw, the compiler is willing to put your values into optionals to make them fit. It can convert i!, an Int, to be of type Int?, and that would match what j is. Since that’s the only possible way this would work, it does it. I’m guessing it rewrites the expression as:

let x = i != nil ? Optional(i!) : j

This looks similar to what was happening with ?? – the left-hand side is getting upconverted from an Int to an Int?. But in this case, it’s happening after the comparison to nil, whereas in the ?? function, it was upconverted beforehand. Which explains why, with the ternary statement version, j is picked over i.

Of course, none of this is likely the behaviour you wanted. You probably just fat-fingered an optional onto the right-hand side, meaning for it to be a regular Int. If so, you might find being explicit about the type would help avoid this confusion:

// if j is an Int?, this will not compile:
let x: Int = i != nil ? i! : j

// and neither will this:
let y: Int = i ?? j

In the case of the ?? version, the compiler’s error message is pretty helpful – it asks you if you meant to have type Int? for j, and offers to stick a ! after it to unwrap it for you. This makes it clear that, in both cases, passing an optional on the right-hand side is user error (which is why I don’t think the way ?? behaves is necessarily a bug).

Finally, here’s a silly thing to try. Up until now, we’ve been fixing the type of the result by converting one of the two types to another. What if you had two types, both of which could be implicitly converted to a third type? Would that work too?

Yes it would. We can create a class that can be constructed from either a string literal or an integer literal, and then use a ternary statement that returned one or the other:

struct Both: IntegerLiteralConvertible, StringLiteralConvertible {
    typealias ExtendedGraphemeClusterLiteralType = String
    let str: String

    static func convertFromIntegerLiteral(value: IntegerLiteralType) -> Both {
        return Both(str: String(value))
    static  func convertFromStringLiteral(value: StringLiteralType) -> Both {
        return Both(str: value)
    static func convertFromExtendedGraphemeClusterLiteral(value: ExtendedGraphemeClusterLiteralType) -> Both {
        return Both(str: value)

let b = true
// x will be of type String
let x = b ? "one" : "two"
// y will be of type Int
let y = b ? 1 : 2
// z1 and z2 will be of type Both
let z1 =  b ? "one" : 2  // {str "one"}
let z2 = !b ? "one" : 2  // {str "2"}

Well, it amused me anyway.

Yo, dawg

EDIT: the behaviour of ?? has been altered as of Swift beta 6. There is now a special case for both the LHS and RHS being of T?, that matches the ternary version. However, the below is still of interest for details of implementing generic functions that take optionals.

The nil coalescing operator (a ?? b) is described in the Swift docs as shorthand for the following code:

a != nil ? a! : b

That is, if a is nil you get b, otherwise you get the unwrapped a.

Ok great, that’s pretty intuitive and very useful. You might be surprised then if you try the following:

let i: Int? = nil
let j: Int? = 5

// i is nil, so this evalates
// to j i.e. {Some 5}
i != nil ? i! : j

// but this returns nil
i ?? j

Why is it returning nil? Is the analogy with the ?: operator a simplification of what’s actually happening? Let’s see, by implementing our own version of the ?? operator, using the exact same ternary logic. Here it is: 1

infix operator ~~ {
    associativity right
    precedence 110

func ~~<T>(a: T?, b: @autoclosure () -> T) -> T {
    return a != nil ? a! : b()

// nope, still nil
i ~~ j

Hum. What’s going on?

Well, to be fair, this isn’t playing by the rules for ??. The docs also state “The expression b must match the type that is stored inside a.” Instead, I’ve passed the same type for both a and b, so the results are undefined.

What is actually happening is that the compiler is fixing the type of T to be the type of the argument on the right-hand side. So T is an Int?. That means the left-hand side T? is an Int??, or to write it longhand, an Optional of an Optional Int. 2

Then, in the call, i is being “upgraded” from an optional to an optional optional. This upconversion is a feature that allows you to pass in plain values to functions that take optional arguments, and have them be automatically wrapped in an optional, rather than having to manually construct an optional to pass in yourself.

This is useful when defining a function with default arguments (arguments which are, ahem, optional). For example the dump command takes a second parameter name: that you can choose not to pass in, that will prefix your dumped object data if you do. An implementation could look like this:

func mydump<T>(x: T, name: String? = nil) {
    if let name = name {
        // if the caller supplied a name, use it
        print(name); print(": ")

// this won't prefix the dump with a name:

// you could call mydump like this:
mydump(a, name: Optional("My object"))

// but there's no need, you can just do this:
mydump(a, name: "My object")

So back to our operator example. With the left-hand side being upconverted, by the time we’re inside our ~~ function, i has become Optional(i). Even if i == nil, Optional(i) != nil. It contains a value (of an optional that doesn’t contain a value).

To simulate what ?? is doing outside of a function, we can do this:

let ii = Optional(i)

// this actually evaluates to nil
ii != nil ? ii! : j

That an optional that contains a nil optional is not equal to nil is pretty important. If it were equal to nil, then iterating over sequences would be a problem. In the following example, the last two entries wouldn’t be reached:

let a: [Int?] = [1, 2, nil, 3, 4]
// remember, is short for the following:
var g = a.generate()
// while the result of next() isn't nil
while let i = {
    // i is now an optional, that will
    // contain nil on the 3rd iteration

Is this behaviour of ?? a bug? I dunno, probably not. You could prefer it to behave like the raw ternary operator, or fail to compile by somehow mandating the right-hand type really be what’s contained in the left-hand optional. But you could also say it’s behaving correctly, based on how the language works, and you might even need it to behave this way in some scenarios.

Either way, it’s a useful case study if you plan on implementing a generic function that takes optionals yourself.

  1. If you’re unclear on why the right-hand side is being declared with an autoclosure, read this post 
  2. I would have loved to put some angle brackets in there, but WordPress had other ideas. It certainly knows how to re-capitalize its name when I get that wrong, though!