Commit Graph

165 Commits

Author SHA1 Message Date
Dirkjan Ochtman be0f8c0ed7 Don't normalize input strings implicitly 2021-02-08 15:53:24 +01:00
Dirkjan Ochtman 8c08bb9e14 Add check_segments function 2021-02-04 11:20:49 +01:00
Dirkjan Ochtman f3aaaa656d Bump version to 0.4.0 2021-02-04 11:20:49 +01:00
Dirkjan Ochtman 5127aac1ec Add optional support for serde 2021-02-04 11:20:49 +01:00
Dirkjan Ochtman bacf82c8cc Separate incorrect segmentation out of TEST_CASES 2021-02-04 10:40:45 +01:00
Dirkjan Ochtman 96187965b6 Extract public asssert_segments() function 2021-02-04 10:40:04 +01:00
Dirkjan Ochtman 45e569379c Default to calculating total from unigram map 2021-02-04 10:36:30 +01:00
Dirkjan Ochtman 0d2930c408 Add API to create segmenter from hashmaps directly 2021-02-04 10:36:30 +01:00
Dirkjan Ochtman b85fc6adc2 Rename testcases to test_cases 2021-02-04 10:36:30 +01:00
Dirkjan Ochtman 55cc7c54a3 Use powi() instead of powf() for performance 2021-02-04 10:17:11 +01:00
Dirkjan Ochtman 970caeba44 Use std HashMap to simplify API 2021-02-04 10:16:38 +01:00
Dirkjan Ochtman c1068c2e53 Bump version number to 0.3.2 2021-02-01 17:25:55 +01:00
Dirkjan Ochtman 29d2d94a8d Reorganize tests and test data to expose test cases 2021-02-01 17:25:32 +01:00
dependabot-preview[bot] d4df4ce29a Update ahash requirement from 0.6.1 to 0.7.0
Updates the requirements on [ahash](https://github.com/tkaitchuck/ahash) to permit the latest version.
- [Release notes](https://github.com/tkaitchuck/ahash/releases)
- [Commits](https://github.com/tkaitchuck/ahash/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2021-01-27 15:30:34 +01:00
Dirkjan Ochtman 41fb2075a6 Tighten the language a little bit 2020-12-16 10:48:31 +01:00
Dirkjan Ochtman 27d20f07e5 Add crate badges to README 2020-12-16 10:44:56 +01:00
Dirkjan Ochtman a8d93efbb6 Add cover to README 2020-12-16 10:42:35 +01:00
Dirkjan Ochtman f51a6e6cd5 Merge branch 'local' into main 2020-12-16 10:41:59 +01:00
Dirkjan Ochtman 275c3c63cb Rename crate to instant-segment 2020-12-16 10:36:38 +01:00
Dirkjan Ochtman 3a37893e74
Update README with new name 2020-12-15 21:02:22 +01:00
Dirkjan Ochtman dcc1c5edc1 Bump version to 0.3.1 2020-12-07 16:30:23 +01:00
Dirkjan Ochtman cb3c9707ef Add docstring for Segmenter type 2020-12-07 14:51:10 +01:00
Dirkjan Ochtman adf7995adb Remove now unused error type 2020-12-07 14:51:10 +01:00
Dirkjan Ochtman 2ab57ca0b1 Fix typo 2020-12-07 14:36:59 +01:00
Dirkjan Ochtman c571996925 Simplify bigram scoring algorithm 2020-12-07 14:24:33 +01:00
Dirkjan Ochtman f26793379b No longer need a macro for testing 2020-12-07 11:55:27 +01:00
Dirkjan Ochtman 912e6477e3 Fix clippy problems in test data setup 2020-12-07 11:46:42 +01:00
Dirkjan Ochtman 4aaa661328 Bump version number to 0.3.0 2020-12-07 11:40:09 +01:00
Dirkjan Ochtman eeb9c77bc7 Simplify Segmenter setup API 2020-12-07 11:39:49 +01:00
Dirkjan Ochtman 4b7760ee41 Bump version number 2020-11-26 11:40:03 +01:00
Dirkjan Ochtman c11da266aa Update performance claim in README 2020-11-26 11:39:57 +01:00
Dirkjan Ochtman d554825594 Name complex type as suggested by clippy 2020-11-26 11:33:36 +01:00
Dirkjan Ochtman 691ecbc3c6 Simplify handling of empty tails 2020-11-26 11:20:06 +01:00
Dirkjan Ochtman ae3896b47b Use range for previous argument as well 2020-11-26 11:15:27 +01:00
Dirkjan Ochtman bc20e39c1e Make slicing cheaper by adding a little unsafe code 2020-11-26 11:14:53 +01:00
Dirkjan Ochtman bb1b1db9c5 Pass Range instead of str to search() 2020-11-26 11:13:35 +01:00
Dirkjan Ochtman 4be435e0fb Make split values absolute instead of relative 2020-11-26 11:12:52 +01:00
Dirkjan Ochtman b7daaff47a Simplify top-level loop 2020-11-26 10:46:27 +01:00
Dirkjan Ochtman 2f9cb95b5c Avoid allocations for split vectors 2020-11-26 10:46:23 +01:00
Dirkjan Ochtman 62bb7130b6 Switch to main as default branch 2020-11-25 22:31:27 +01:00
Dirkjan Ochtman bd497d155d Remove unnecessary closure wrapper 2020-11-25 22:30:14 +01:00
Dirkjan Ochtman ed8e13b10c Use all features for clippy, too 2020-11-25 22:24:08 +01:00
Dirkjan Ochtman 52dd5ec598 Allow MPL licenses 2020-11-25 17:34:03 +01:00
Dirkjan Ochtman a1f03e32fe Remove unused lifetime 2020-11-25 17:33:50 +01:00
Dirkjan Ochtman 2164044eb1 Build with all features enabled 2020-11-25 17:31:35 +01:00
Dirkjan Ochtman 47271ff81e Allocate a single Vec to back cached splits 2020-11-25 17:29:13 +01:00
Dirkjan Ochtman 947e003a48 Store splits instead of string slices 2020-11-25 17:29:13 +01:00
Dirkjan Ochtman 1df3c4397e Inline TextDivider iterator 2020-11-25 17:29:13 +01:00
Dirkjan Ochtman ead9a3064b Better typed handling of previous word 2020-11-25 17:29:13 +01:00
Dirkjan Ochtman ea4438f2e8 Make Segmenter::score() slightly more efficient 2020-11-25 17:29:13 +01:00