The Wayback Machine - https://web.archive.org/web/20201020112452/https://github.com/go-sql-driver/mysql/pull/667
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ColumnType interfaces #667

Merged
merged 25 commits into from Oct 17, 2017
Merged

ColumnType interfaces #667

merged 25 commits into from Oct 17, 2017

Conversation

@julienschmidt
Copy link
Member

@julienschmidt julienschmidt commented Sep 18, 2017

Description

Fixes #495

Checklist

  • Code compiles correctly
  • Created tests which fail without the change (if possible)
  • All tests passing
  • Extended the README / documentation, if necessary
  • Added myself / the copyright holder to the AUTHORS file
@julienschmidt julienschmidt added this to the v1.4 milestone Sep 18, 2017
@julienschmidt julienschmidt self-assigned this Sep 18, 2017
@julienschmidt julienschmidt mentioned this pull request Sep 18, 2017
3 of 5 tasks complete
@julienschmidt
Copy link
Member Author

@julienschmidt julienschmidt commented Sep 18, 2017

We should probably add sanity-checks for the index, as in #664

fields.go Outdated
fieldTypeMediumBLOB: "MEDIUMBLOB",
fieldTypeLongBLOB: "LONGBLOB",
fieldTypeBLOB: "BLOB",
fieldTypeVarString: "VARSTRING", // correct?

This comment has been minimized.

@arnehormann

arnehormann Sep 18, 2017
Member

I didn't see these in MySQL anywhere. The docs, e.g. https://dev.mysql.com/doc/refman/5.7/en/string-types.html don't mention them either. I don't think they are correct, should probably be varchar.

This comment has been minimized.

@methane

methane Sep 19, 2017
Member

https://github.com/PyMySQL/PyMySQL/blob/e027812ab0eab4d0557fa2ab66520d5eb38fc28f/pymysql/connections.py#L1502-L1507

There are no document about it.
After seeing some response, I decided to handle charsetnr == 63 is bytes, not string.

This comment has been minimized.

@methane

methane Sep 19, 2017
Member

Of course, string can contain bytes.

rows.go Outdated
}

func (rows *mysqlRows) ColumnTypeNullable(i int) (nullable, ok bool) {
return rows.rs.columns[i].flags&flagNotNULL == 0, true

This comment has been minimized.

@arnehormann

arnehormann Sep 18, 2017
Member

If we add the sanity checks here, it should also check for a negative index for ok.

@@ -87,8 +87,10 @@ const (
)

// https://dev.mysql.com/doc/internals/en/com-query-response.html#packet-Protocol::ColumnType
type fieldType byte

This comment has been minimized.

@arnehormann

arnehormann Sep 18, 2017
Member

I'm not convinced this helps taking all the casting happening in packets.go ...

@julienschmidt julienschmidt force-pushed the columntype branch from 01e7b59 to 925d497 Sep 29, 2017
@julienschmidt julienschmidt force-pushed the columntype branch from 925d497 to 163ddcd Oct 3, 2017
@julienschmidt
Copy link
Member Author

@julienschmidt julienschmidt commented Oct 7, 2017

Removed the ColumnTypeLength implementation for now and added a section to the README

@arnehormann
Copy link
Member

@arnehormann arnehormann commented Oct 7, 2017

@julienschmidt
Copy link
Member Author

@julienschmidt julienschmidt commented Oct 7, 2017

That's what I assumed. There is also a character_set field for every column right before the length.
But do we really want to keep track of the factor for every single charset? Is it even possible or do some charsets have a non-constant factor?

We can work on this later, but for now I'd like to get this PR merged without the length before it gets too big.

@methane
Copy link
Member

@methane methane commented Oct 12, 2017

but for now I'd like to get this PR merged without the length before it gets too big.

Completely agree.

@methane
Copy link
Member

@methane methane commented Oct 16, 2017

Code looks OK.
But I don't know much about real world usage of the API.
Should we distinguish BLOB and TEXT?

But maybe, it can be implemented later.

@julienschmidt
Copy link
Member Author

@julienschmidt julienschmidt commented Oct 17, 2017

How would you distinguish BLOB and TEXT?
AFAIK we currently treat both just as a sequence of bytes on the driver level.

And regarding usage: Wherever not all required information about the db schema is known at compile-time. Think for example of ORMs or applications like phpMyAdmin (goMyAdmin?).

@methane
Copy link
Member

@methane methane commented Oct 17, 2017

How would you distinguish BLOB and TEXT?

charsetnr can be used for it. It is 63 for binary columns.
https://github.com/PyMySQL/PyMySQL/blob/dc01c6e7cd06cf523d67140b26afea365c130192/pymysql/connections.py#L1502-L1507

@julienschmidt
Copy link
Member Author

@julienschmidt julienschmidt commented Oct 17, 2017

I mean, how would we treat it differently?

@methane
Copy link
Member

@methane methane commented Oct 17, 2017

DatabaseType() returns TEXT or VARCHAR instead of BLOB or VARBINARY.

@molon

This comment has been minimized.

Copy link

@molon molon commented on 4023d9a Oct 17, 2017

Hello, thanks for the pr. It saved me so long time.
But why do this commit?
I cant distinguish the scanTypeString now. So many type is RawBytes now.

This comment has been minimized.

Copy link
Member Author

@julienschmidt julienschmidt replied Oct 17, 2017

It only removed unused code.
So many types are RawBytes because that is what the MySQL protocol transmits in many cases (text) and that is also what the driver returns then.

@julienschmidt
Copy link
Member Author

@julienschmidt julienschmidt commented Oct 17, 2017

@methane I'd suggest we implement that in another PR. The MySQL server also seems to treat them the same internally.
I'd also replace the map with a case-switch construct, in where it would be easier to making this further differentiation.

@methane
Copy link
Member

@methane methane commented Oct 17, 2017

I agree with you.
We can change it after see feedback from users.

LGTM for now.

@julienschmidt julienschmidt merged commit c6c4e3c into master Oct 17, 2017
3 checks passed
3 checks passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.5%) to 77.177%
Details
@julienschmidt julienschmidt deleted the columntype branch Oct 17, 2017
fields.go Outdated
return scanTypeString
}
return scanTypeNullString
return scanTypeRawBytes

This comment has been minimized.

@molon

molon Oct 17, 2017

Do I have to check nullable to decide converting to sql.NullString or string myself?

This comment has been minimized.

@julienschmidt

julienschmidt Oct 17, 2017
Author Member

sql.RawBytes can also be nil, which is distinguishable from an empty string (""). I.e. if rawBytes == nil then it was a NULL value.
If you want to known if the column itself may contain NULL values, then you should check ColumnType.Nullable()

bLamarche413 pushed a commit to bLamarche413/mysql that referenced this pull request Mar 23, 2018
* rows: implement driver.RowsColumnTypeScanType

Implementation for time.Time not yet complete!

* rows: implement driver.RowsColumnTypeNullable

* rows: move fields related code to fields.go

* fields: use NullTime for nullable datetime fields

* fields: make fieldType its own type

* rows: implement driver.RowsColumnTypeDatabaseTypeName

* fields: fix copyright year

* rows: compile time interface implementation checks

* rows: move tests to versioned driver test files

* rows: cache parseTime in resultSet instead of mysqlConn

* fields: fix string and time types

* rows: implement ColumnTypeLength

* rows: implement ColumnTypePrecisionScale

* rows: fix ColumnTypeNullable

* rows: ColumnTypes tests part1

* rows: use keyed composite literals in ColumnTypes tests

* rows: ColumnTypes tests part2

* rows: always use NullTime as ScanType for datetime

* rows: avoid errors through rounding of time values

* rows: remove parseTime cache

* fields: remove unused scanTypes

* rows: fix ColumnTypePrecisionScale implementation

* fields: sort types alphabetical

* rows: remove ColumnTypeLength implementation for now

* README: document ColumnType Support
carlosms added a commit to carlosms/gitbase-playground that referenced this pull request Apr 25, 2018
Latest release for mysql driver, 1.3, does not support
ColumnType.DatabaseTypeName. See
go-sql-driver/mysql#667

Signed-off-by: Carlos Martín <[email protected]>
carlosms added a commit to carlosms/gitbase-playground that referenced this pull request Apr 25, 2018
Latest release for mysql driver, 1.3, does not support
ColumnType.DatabaseTypeName. See
go-sql-driver/mysql#667

Signed-off-by: Carlos Martín <[email protected]>
carlosms added a commit to carlosms/gitbase-playground that referenced this pull request Apr 25, 2018
Latest release for mysql driver, 1.3, does not support
ColumnType.DatabaseTypeName. See
go-sql-driver/mysql#667

Signed-off-by: Carlos Martín <[email protected]>
carlosms added a commit to carlosms/gitbase-playground that referenced this pull request Apr 27, 2018
Latest release for mysql driver, 1.3, does not support
ColumnType.DatabaseTypeName. See
go-sql-driver/mysql#667

Signed-off-by: Carlos Martín <[email protected]>
carlosms added a commit to carlosms/gitbase-playground that referenced this pull request May 3, 2018
Latest release for mysql driver, 1.3, does not support
ColumnType.DatabaseTypeName. See
go-sql-driver/mysql#667

Signed-off-by: Carlos Martín <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.