More fun with implicitly wrapped non-optionals

As we’ve seen previously, Swift will happily auto-wrap your non-optional value inside an optional temporarily if it helps make the types involved compatible. We’ve seen the occasional instance where this can be surprising. But for the most part, this implicit wrapping of non-optionals is your friend.1 Here’s a few more examples.

Equating Optionals

The Swift standard library defines an == operator for optionals:

func ==<T : Equatable>(lhs: T?, rhs: T?) -> Bool

So long as your optional wraps a type that’s Equatable, that means you can compare two optionals of the same underlying type:

let i: Int? = 1
let j: Int? = 1
let k: Int? = 2
let m: Int? = nil
let n: Int? = nil

i == j  // true,  1 == 1
i == k  // false, 1 != 2
i == m  // false, non-nil != nil
m == n  // true,  nil == nil

let a: [Int]? = [1,2,3]
let b: [Int]? = [1,2,3]

// error, [Int] is not equatable
// (it does have an == operator, but that's not the same thing)
a == b

let f: Double? = 1.0

// ignore the weird compiler error,
// the problem is Int and Double aren't
// the same type...
i == f

Well, this is useful, but even more useful is this one definition of == allows you to compare optionals against non-optionals too. Suppose you want to know if an optional string contains a specific value. You don’t care if it’s nil or not – just whether it contains that value.

You can just pass your non-optional into that same == that takes optionals:

let x = "Elsa"

let s1: String? = "Elsa"
let s2: String? = "Anna"
let s3: String? = nil

s1 == x  // true
s2 == x  // false
s3 == x  // false

This works because Swift is implicitly wrapping x in an optional each time. If Swift didn’t do this, and you implemented an optional-to-non-optional operator yourself, you’d have to define == an extra two times, to make sure it was symmetric (which is a requirement whenever you implement ==):

// definitions if there weren't implicit optionals
func ==<T:Equatable>(lhs:T, rhs:T?)->Bool { return rhs != nil && lhs == rhs! }
func ==<T:Equatable>(lhs:T?, rhs:T)->Bool { return lhs != nil && lhs! == rhs }

Beware though, optional == isn’t all sunshine and unicorn giggles. Imagine the following code:

let a: Any = "Hello"
let b: Any = "Goodbye"

// false, of course
(a as? String) == (b as? String)

// oops, this is true...
(a as? Int) == (b as? Int)

Depending on how you think, this is either perfectly reasonable or immensley confusing, but either way you should bear it in mind. And maybe it’s another reason to go on your long list “Why not to use Any and as unless you really really have to.”

Comparing Optionals

Unsurprisingly, there’s a similar definition of < that will compare two optionals so long as what they wrap is Comparable:

func <<T : _Comparable>(lhs: T?, rhs: T?) -> Bool

If lhs and rhs are both some value, it compares them. Two nils are considered equal just like before. But a nil is always considered less than any non-nil value, (Comparable must always enforce a strict total order so they have to go somewhere well-defined in the ordering).

So if you sort a set of optionals, the nil values all end up at one end:

let a: [Int?] = [1, 3, nil, 2]
sorted(a, <)
// results in [nil, 1, 2, 3]

This is fairly intuitive, but it can lead to other subtle bugs. Suppose you want to filter a list like so:2

let ages = [
    "Tim":  53,  "Angela": 54,  "Craig":   44,
    "Jony": 47,  "Chris":  37,  "Michael": 34,
]

let people = ["Tim", "Johny", "Chris", "Michael"]

// don't trust anyone over 40
let trust = people.filter { ages[$0] < 40 }

// trust will contain ["Johny", "Chris", "Michael"]

The string "Johny" was not in the ages dictionary so you get a nil – but the way < works, nil is less than 40, so included in the filtered array.

Optional Subscripts

While we're talking about dictionaries, did you know that Dictionary.subscript(key) doesn't just return an optional, it also takes an optional?

var d = ["1":1]

let four = "4"
let nonsense = "nonsense"

// this will add ["4":4] to d
// remember toInt returns an optional
// (the string might not have been a number)
d[four] = four.toInt()

// nothing will be added, because
// the rhs is nil
d[nonsense] = nonsense.toInt()

// this will remove the "1" entry from d
d["1"] = nil

That's quite useful – but also, there wasn't much choice in it working like this.3 Dictionary.subscript(key) { set } has to take an optional, because when you define a setter, you must define a getter. And Dictionary's getter has to be optional (in case the key isn't present). So the setter's input value has to be.

This means every time you assign a value to a dictionary using a key subscript, your non-optional value is getting auto-wrapped in an optional. If this didn’t happen automatically, it would make assigning values to your dictionary a real pain, as you’d have to type d[key] = .Some(value) every time.

To see this in practice, here's a very bare-bones (very inefficient) implementation of a dictionary. Take a look at the subscript to see how the optional is handled:

public struct RubbishDictionary<Key: Equatable, Value> {

    public typealias Element = (Key, Value)

    // just store the key/value pairs in an unsorted
    // array (see, told you it was rubbish)
    private typealias Storage = [Element]
    private var _store: Storage = []

    // there's a reason why this is a private method,
    // which I'll get to later...
    private func _indexForKey(key: Key) -> Storage.Index? {
        // new year's resolution: see if I can avoid
        // calling array[i] completely in 2015
        for (idx, (k, _)) in Zip2(indices(_store),_store) {
            if key == k { return idx }
        }
        return nil
    }

    // Subscript for key-based lookup/storage
    public subscript(key: Key) -> Value? {
        // "get" is pretty simple, either key can be
        // found, or not...
        get {
            if let idx = _indexForKey(key) {
                // implicit optional wrap
                return _store[idx].1
            }
            else {
                return nil
            }
        }
        // remember, newValue is a Value?
        set(newValue) {
            switch (_indexForKey(key), newValue) {

            // existing index, so replace value
            case let (.Some(idx), .Some(value)):
                _store[idx].1 = value

            // existing index, nil means remove value
            case let (.Some(idx), .None):
                _store.removeAtIndex(idx)

            // new index, add non-nil value
            case let (.None, .Some(value)):
                _store.append((key, value))

            // new index, nil value is no-op
            case (.None,.None):
                break
            }
        }
    }
}

Finally, just a reminder that an optional containing an optional containing nil is not the same as nil:

var d: [Int:Int?] = [1:1, 2:nil]
// d now contains two entries, 1:1 and 2:nil
d[2] = nil  // removes nil
d[3] = .Some(nil)  // adds 3:nil

Full imlementation of RubbishDictionary

So, in case you've made it all the way down here, and you're interested, I thought I'd end with a fuller but minimal-as-possible implementation of RubbishDictionary. It has almost-feature-parity with Dictionary (it's just missing getMirror and init(minimumCapatity)). It ought to do everything Dictionary can, just slower.

If reading raw code's not your thing, skip it. But it shows how to simply implement several of the standard library's protocols such as CollectionType and DictionaryLiteralConvertible, as well as a few other features like lazily-evaluated key and value collections. I like how Swift extensions allow you to build up the type in logical groups (e.g. each protocol implementation one-by-one). Since this is a post about implicit optional wrapping, I've flagged where it’s also helping keep the code consise.

I have a nagging feeling there are some neater ways to handle some of the optionals – if you spot one, let me know. Oh and bugs. I'm sure there are a couple of them.

// building on the initial definition given above...

// An index type for RubbishDictionary.  In theory, you could perhaps just
// expose an alias of the storage type’s index, however, if the Key type were
// an Int, this would mean subscript(key) and subscript(index) would be
// ambiguous. So instead it's simple wrapper struct that does little more than
// pass-through to the real index.
public struct RubbishDictionaryIndex<Key: Equatable, Value>: BidirectionalIndexType {
    private let _idx: RubbishDictionary<Key,Value>.Storage.Index

    public typealias Index = RubbishDictionaryIndex<Key,Value>

    public func predecessor() -> Index { return Index(_idx: _idx.predecessor()) }
    public func successor() -> Index { return Index(_idx: _idx.successor()) }
}

// Required to make the index conform to Equatable
// (indirectly required by BidirectionalIndexType)
public func ==<Key: Equatable, Value>
  (lhs: RubbishDictionaryIndex<Key,Value>, rhs: RubbishDictionaryIndex<Key,Value>)
   -> Bool { 
    return lhs._idx == rhs._idx 
}

// Conformance to CollectionType
extension RubbishDictionary: CollectionType {
    public typealias Index = RubbishDictionaryIndex<Key,Value>

    // Add a subscript that takes an Index.  This one
    // does not need to be nil, since an indexed entry
    // _must_ exist.  Read-only.
    public subscript(idx: Index) -> Element { return _store[idx._idx] }

    public var startIndex: Index { return Index(_idx: _store.startIndex) }
    public var endIndex: Index { return Index(_idx: _store.endIndex) }

    public typealias Generator = IndexingGenerator<RubbishDictionary>
    public func generate() -> Generator { return Generator(self) }
}

// Conformance to Printable and DebugPrintable
extension RubbishDictionary: Printable, DebugPrintable {
    public var description: String {
        let elements = map(self) { "($0.0):($0.1)" }
        return "[" + ",".join(elements) + "]"
    }

    public var debugDescription: String {
        let elements = map(self) { "(toDebugString($0.0)):(toDebugString($0.1))" }
        return "[" + ",".join(elements) + "]"
    }
}

// Conformance to DictionaryLiteralConvertible
extension RubbishDictionary: DictionaryLiteralConvertible {
    public init(dictionaryLiteral elements: (Key, Value)...) {
        for (k,v) in elements {
            self[k] = v
        }
    }
}

// Remaining features supported by Dictionary
extension RubbishDictionary {

    public var count: Int { return _store.count }
    public var isEmpty: Bool { return _store.isEmpty }

    // now for the public version of indexForKey
    public func indexForKey(key: Key) -> Index? {
        // this maps non-nil results from _indexForKey to
        // the wrapper type, or just returns nil if 
        // _indexForKey did...
        return _indexForKey(key).map { Index(_idx: $0) }
    }

    // a lazily-evaluated collection of keys in the dictionary
    public var keys: LazyBidirectionalCollection<MapCollectionView<RubbishDictionary<Key,Value>,Key>> {
        return lazy(self).map { (k,v) in k }
    }

    // and the same for the values
    public var values: LazyBidirectionalCollection<MapCollectionView<RubbishDictionary<Key,Value>,Value>> {
        return lazy(self).map { (k,v) in v }
    }

    // unlike its subscript equivalent, updateValue:forKey does not take an optional value...
    public mutating func updateValue(value: Value, forKey key: Key) -> Value? {
        if let idx = _indexForKey(key) {
            let old = _store[idx].1
            _store[idx].1 = value
            // implicit optional wrap
            return old
        }
        else {
            _store.append((key,value))
            return nil
        }
    }

    // returns the value removed, if there was one
    public mutating func removeValueForKey(key: Key) -> Value? {
        if let idx = _indexForKey(key) {
            // implicit optional wrap
            return _store.removeAtIndex(idx).1
        }
        else {
            return nil
        }
    }

    public mutating func removeAtIndex(index: Index) {
        _store.removeAtIndex(index._idx)
    }

    public mutating func removeAll(keepCapacity: Bool = false) { 
        _store.removeAll(keepCapacity: keepCapacity)
    }

}

  1. Unlike implicitly-wrapped optionals, which are more like the kind of friend that gets you into fights in bars. 
  2. I'm getting a lot of mileage out of this example
  3. I guess it could have asserted on a nil value. 

Implicitly converting functions to return optionals

One final post on upconversion of non-optional values to optional. As we saw previously, if you pass a non-optional value to a function argument that expects an optional, it will get automatically converted into an optional by the compiler.

Ken Ferry points out something that goes one step further. If a function takes an argument of a function that returns an optional, and you pass into it a function that does not return an optional, the function will automatically be converted to return an optional.

That is:

func foo()->Int {
    return 1
}

func bar(fun: ()->Int?) {
    println(fun())
}

// Even though foo returns an Int, 
// you can pass it to bar.
// Inside bar it will return an Int?,
// so this prints "Optional(1)"
bar(foo)

What’s more, you can even store or return the converted function:

// alter bar to return the passed-in function
func bar(fun: ()->Int?)->()->Int? {
    return fun
}

var i = 0
let f = { ++i }

f() // returns 1
f() // returns 2

let g = bar(f)

g() // returns {Some 3}
g() // returns {Some 4}
f() // returns 5

This feature makes sense when you think about it in the context of other implicit conversions – if you can pass non-optional values in to functions and have them implicitly converted to optionals, the logical next step is to be able to do the same for the return value of functions you pass in as parameters.

This only works on the return value of the function, though. The following won’t compile:

func foo(i: Int)->Int {
    return i
}

func bar(fun: (Int?)->Int?) {
    println(fun(1))
}

// won't compile - no implicit conversion
// of foo's argument to an Int?
bar(foo)

This is because foo is expecting a non-nil argument – it wouldn’t know how handle nil if it received one. However, what would compile is if foo could handle an optional, but bar was expecting a function with a non-optional argument:

func foo(i: Int?)->Int {
    return i ?? 0
}

func bar(fun: (Int)->Int?) {
    println(fun(1))
}

// will compile - implicit conversion
// of foo's argument to an Int
bar(foo)

This works because any call by bar of foo could convert the non-optional parameter it passes in to an optional implicitly. (thanks to @westacular and @jvasileff for pointing this out)

Just like you can think of passing non-optionals into optional arguments as the compiler automatically wrapping your values in an optional, you can think of the compiler silently wrapping your non-optional-returning function in a closure that calls your function and returns its value wrapped in an optional:

func foo()->Int {
    return 1
}

func bar(fun: ()->Int?) {
    println(fun())
}

// to do an explicit equivalent 
// of the compiler's conversion:
bar( { Optional(foo()) } )

In practice, the compiler probably does something a bit more low-level. If you wrote the above code, and then stepped through it with the debugger, and you put a breakpoint inside foo, you’d see an extra entry in the stack trace between foo and bar showing the in-between closure. But if you leave the compiler to do it implicitly, you’ll see no such entry.

If you’re interested in digging a bit further into exactly where the compiler is doing the conversion, you can use a feature of the Swift compiler to dump the syntax tree. Here is a dump from the first piece of code in this article:1

% xcrun swiftc -dump-ast main.swift
(source_file
  (func_decl "foo()" type='() -> Int' access=internal
    (body_params
      (pattern_tuple type='()'))
    (result
      (type_ident
        (component id='Int' bind=type)))
    (brace_stmt
      (return_stmt
        (call_expr implicit type='Int' location=main.swift:2:12 range=[main.swift:2:12 - line:2:12]
          (dot_syntax_call_expr type='(Int2048) -> Int' location=main.swift:2:12 range=[main.swift:2:12 - line:2:12]
            (declref_expr implicit type='Int.Type -> (Int2048) -> Int' location=main.swift:2:12 range=[main.swift:2:12 - line:2:12] decl=Swift.(file).Int._convertFromBuiltinIntegerLiteral specialized=no)
            (type_expr implicit type='Int.Type' location=main.swift:2:12 range=[main.swift:2:12 - line:2:12] typerepr='<<IMPLICIT>>'))
          (integer_literal_expr type='Int2048' location=main.swift:2:12 range=[main.swift:2:12 - line:2:12] value=1)))))
  (func_decl "bar(_:)" type='(() -> Int?) -> ()' access=internal
    (body_params
      (pattern_tuple type='(fun: () -> Int?)'
        (pattern_typed type='() -> Int?'
          (pattern_named type='() -> Int?' 'fun')
          (type_function
            (type_tuple)
))))
    (brace_stmt
      (call_expr type='()' location=main.swift:6:5 range=[main.swift:6:5 - line:6:18]
        (declref_expr type='(Int?) -> ()' location=main.swift:6:5 range=[main.swift:6:5 - line:6:5] decl=Swift.(file).println [with T=Int?] specialized=no)
        (paren_expr type='(Int?)' location=main.swift:6:13 range=[main.swift:6:12 - line:6:18]
          (call_expr type='Int?' location=main.swift:6:13 range=[main.swift:6:13 - line:6:17]
            (declref_expr type='() -> Int?' location=main.swift:6:13 range=[main.swift:6:13 - line:6:13] decl=main.(file).func decl.fun@main.swift:5:10 specialized=no)
            (tuple_expr type='()' location=main.swift:6:16 range=[main.swift:6:16 - line:6:17]))))))
  (top_level_code_decl
    (brace_stmt
      (call_expr type='()' location=main.swift:9:1 range=[main.swift:9:1 - line:9:8]
        (declref_expr type='(() -> Int?) -> ()' location=main.swift:9:1 range=[main.swift:9:1 - line:9:1] decl=main.(file).bar@main.swift:5:6 specialized=no)
        (paren_expr type='(() -> Int?)' location=main.swift:9:5 range=[main.swift:9:4 - line:9:8]
          (function_conversion_expr implicit type='() -> Int?' location=main.swift:9:5 range=[main.swift:9:5 - line:9:5]
            (declref_expr type='() -> Int' location=main.swift:9:5 range=[main.swift:9:5 - line:9:5] decl=main.(file).foo@main.swift:1:6 specialized=no))))))

This is fairly simple to tie back to the original code. We have three sections under source_file: two func_decls, one for foo and one for bar, followed by a top_level_code_decl showing the call to bar passing in foo. And we see that the compiler inserts a function_conversion_expr implicit type='() ->Int?' (line 36).

Incidentally, if you run -dump-ast on a call that requires a conversion from regular type like Int to an optional, you’d see inject_into_optional implicit type='Int?'. And for an implicit conversion to an Any type, you’d see erasure_expr implicit type='Any'.


  1. swiftc dumps to stderr. If you want to pipe longer examples through less, and like me you need to trial-and-error where the ampersand goes every damn time despite how many thousand times you might have typed it before, it’s 2>&1 

Bachman Ternary Overdrive

(ok I think I’ve pushed the puns too far with that headline)

David Owens rightly called me out for pulling a bit of a fast one at the start of my last article on how ?: and ?? differ in behaviour. Not that they don’t, but I was acting as if what the ternary operator version was doing was something perfectly sensible, and that ?? was the funny one. But I glossed over what the ternary operator itself was doing.

Look at this code again:

let i: Int? = nil
let j: Int? = 5
let k: Int? = 6

// this returns {Some 5}
let x = i != nil ? i! : j
// this returns {Some 6}
let y = k != nil ? k! : j

What are the types of x and y? Int?, right? And how is that decided? From the type of the ternary expression. But which part? If you look at the second and third part of the expression, they’re different types. i! and k! are of type Int, whereas j is of type Int?.

That’s no good! Both possible values from the ternary expression have to be able to become the same type, because the type of x is determined at compile time, before you know what i or k contains.

For example, this won’t compile:

let s = "hello"
let i = 123
// x can't be either a String or an Int
let x = i > 0 ? s : i

But note I said become the same type, not be the same type. As we saw, the compiler is willing to put your values into optionals to make them fit. It can convert i!, an Int, to be of type Int?, and that would match what j is. Since that’s the only possible way this would work, it does it. I’m guessing it rewrites the expression as:

let x = i != nil ? Optional(i!) : j

This looks similar to what was happening with ?? – the left-hand side is getting upconverted from an Int to an Int?. But in this case, it’s happening after the comparison to nil, whereas in the ?? function, it was upconverted beforehand. Which explains why, with the ternary statement version, j is picked over i.

Of course, none of this is likely the behaviour you wanted. You probably just fat-fingered an optional onto the right-hand side, meaning for it to be a regular Int. If so, you might find being explicit about the type would help avoid this confusion:

// if j is an Int?, this will not compile:
let x: Int = i != nil ? i! : j

// and neither will this:
let y: Int = i ?? j

In the case of the ?? version, the compiler’s error message is pretty helpful – it asks you if you meant to have type Int? for j, and offers to stick a ! after it to unwrap it for you. This makes it clear that, in both cases, passing an optional on the right-hand side is user error (which is why I don’t think the way ?? behaves is necessarily a bug).

Finally, here’s a silly thing to try. Up until now, we’ve been fixing the type of the result by converting one of the two types to another. What if you had two types, both of which could be implicitly converted to a third type? Would that work too?

Yes it would. We can create a class that can be constructed from either a string literal or an integer literal, and then use a ternary statement that returned one or the other:

struct Both: IntegerLiteralConvertible, StringLiteralConvertible {
    typealias ExtendedGraphemeClusterLiteralType = String
    let str: String

    static func convertFromIntegerLiteral(value: IntegerLiteralType) -> Both {
        return Both(str: String(value))
    }
    static  func convertFromStringLiteral(value: StringLiteralType) -> Both {
        return Both(str: value)
    }
    static func convertFromExtendedGraphemeClusterLiteral(value: ExtendedGraphemeClusterLiteralType) -> Both {
        return Both(str: value)
    }
}

let b = true
// x will be of type String
let x = b ? "one" : "two"
// y will be of type Int
let y = b ? 1 : 2
// z1 and z2 will be of type Both
let z1 =  b ? "one" : 2  // {str "one"}
let z2 = !b ? "one" : 2  // {str "2"}

Well, it amused me anyway.

Yo, dawg

EDIT: the behaviour of ?? has been altered as of Swift beta 6. There is now a special case for both the LHS and RHS being of T?, that matches the ternary version. However, the below is still of interest for details of implementing generic functions that take optionals.

The nil coalescing operator (a ?? b) is described in the Swift docs as shorthand for the following code:

a != nil ? a! : b

That is, if a is nil you get b, otherwise you get the unwrapped a.

Ok great, that’s pretty intuitive and very useful. You might be surprised then if you try the following:

let i: Int? = nil
let j: Int? = 5

// i is nil, so this evalates
// to j i.e. {Some 5}
i != nil ? i! : j

// but this returns nil
i ?? j

Why is it returning nil? Is the analogy with the ?: operator a simplification of what’s actually happening? Let’s see, by implementing our own version of the ?? operator, using the exact same ternary logic. Here it is: 1

infix operator ~~ {
    associativity right
    precedence 110
}

func ~~<T>(a: T?, b: @autoclosure () -> T) -> T {
    return a != nil ? a! : b()
}

// nope, still nil
i ~~ j

Hum. What’s going on?

Well, to be fair, this isn’t playing by the rules for ??. The docs also state “The expression b must match the type that is stored inside a.” Instead, I’ve passed the same type for both a and b, so the results are undefined.

What is actually happening is that the compiler is fixing the type of T to be the type of the argument on the right-hand side. So T is an Int?. That means the left-hand side T? is an Int??, or to write it longhand, an Optional of an Optional Int. 2

Then, in the call, i is being “upgraded” from an optional to an optional optional. This upconversion is a feature that allows you to pass in plain values to functions that take optional arguments, and have them be automatically wrapped in an optional, rather than having to manually construct an optional to pass in yourself.

This is useful when defining a function with default arguments (arguments which are, ahem, optional). For example the dump command takes a second parameter name: that you can choose not to pass in, that will prefix your dumped object data if you do. An implementation could look like this:

func mydump<T>(x: T, name: String? = nil) {
    if let name = name {
        // if the caller supplied a name, use it
        print(name); print(": ")
    }
    println(x)
}

// this won't prefix the dump with a name:
mydump(a)

// you could call mydump like this:
mydump(a, name: Optional("My object"))

// but there's no need, you can just do this:
mydump(a, name: "My object")

So back to our operator example. With the left-hand side being upconverted, by the time we’re inside our ~~ function, i has become Optional(i). Even if i == nil, Optional(i) != nil. It contains a value (of an optional that doesn’t contain a value).

To simulate what ?? is doing outside of a function, we can do this:

let ii = Optional(i)

// this actually evaluates to nil
ii != nil ? ii! : j

That an optional that contains a nil optional is not equal to nil is pretty important. If it were equal to nil, then iterating over sequences would be a problem. In the following example, the last two entries wouldn’t be reached:

let a: [Int?] = [1, 2, nil, 3, 4]
// remember, for...in is short for the following:
var g = a.generate()
// while the result of next() isn't nil
while let i = g.next() {
    // i is now an optional, that will
    // contain nil on the 3rd iteration
}

Is this behaviour of ?? a bug? I dunno, probably not. You could prefer it to behave like the raw ternary operator, or fail to compile by somehow mandating the right-hand type really be what’s contained in the left-hand optional. But you could also say it’s behaving correctly, based on how the language works, and you might even need it to behave this way in some scenarios.

Either way, it’s a useful case study if you plan on implementing a generic function that takes optionals yourself.


  1. If you’re unclear on why the right-hand side is being declared with an autoclosure, read this post 
  2. I would have loved to put some angle brackets in there, but WordPress had other ideas. It certainly knows how to re-capitalize its name when I get that wrong, though!